"Perspectives and Priorities for Virtual Environments Research"

creepytreatmentAI and Robotics

Nov 14, 2013 (3 years and 11 months ago)

75 views

"Perspectives and Priorities for Virtual
Environments Research"



Professor R. A. Earnshaw

Dept of Electronic Imaging and Media Communications

University of Bradford

Bradford BD7 1DP

UK

R.A.Earnshaw@bradfor
d.ac.uk

http://www.eimc.brad.ac.uk/



Abstract

This paper summarises current trends, developments and issues in virtual reality and outlines areas
needing further research to make the technology more usable by hu
mans or more effective in
serving particular application domains. Recent EPSRC and European/NSF Workshops have sought
to identify the research areas that need to be addressed and these are summarised in this paper.



1. Current Trends, Developments, and I
ssues


1.1 Industrial Applications

Andreas Roessler, Virtual Reality Competence Centre, Fraunhofer, reports the following results from
industrial applications.


Current trends in real world applications are not to use head
-
mounted displays or gloves, but r
ather
immersive or wall
-
based projection technology and simple input devices. Head
-
mounted displays have the
following problems


-

Limited resolution

-

Wires and cables limit human movement

-

Difficult to engage with other users in the real world

-

Ergonomic iss
ues of wearing a helmet


Gloves have the following problems


-

Mean time between faults is low

-

One size has to fit all

-

Complexity


30 values in 30 Hz

-

Unnatural


no force feedback for the hand when “touching” something

-

Lack of comfort and ease of use


Rece
nt experiments with the use of a Powerwall at BMW for CAD evaluation of car shapes has produced the
following results


-

6D user interface is closer to reality, without forces

-

There is a big difference between laboratory trials and productive usage

-

small di
fferences in user interface make a big difference for the user

-

User should drive the process and iterate with the developer


Issues which need addressing




2

Users of Virtual Reality

-

Understanding the nature of VR

-

From technology push to application pull (a
nd better meet the user’s needs)

-

How to learn to use VR (rather than leaving it to the VR experts)

-

User acceptance that it is useful


Interaction

-

Development of minimal user interfaces with high usability (3D mouse?)

-

Standard user interfaces

-

Complex user i
nterfaces (hierarchical are not easy to use)


User Effects

-

Long term usage effects

-

Health and safety issues

-

Visual systems

-

Haptics without mechanical linkages


Technology

-

High level development tools


reusable interaction toolkits

-

PC
-
based Immersive Proje
ction Technology

-

Wireless interaction

-

Precise tracking


1.2 EPSRC

A recent EPSRC Workshop (21
-
22 February 2000) on defining the future research agenda for human factors
of virtual reality identified the following areas where work needed to be done.




More N
atural User Interfaces


Movement and Haptics



Interaction with complex materials and properties is very difficult



Precision movements versus gross movements and rotating objects are difficult



The “haptic glace” conveys a lot of information to the human user

from a brief interaction, but
context affects the sense of touch, which in turn determines the quality of the interaction



Other modalities such as vision affect haptic interaction.


3D audio and speech/natural language processing



3D sound representation h
as applications in submarines, alarms, architectural



Matching speech to the visual appearance of an avatar and embodying emotion



Synchronising the visual with speech when the user gives priority to the visual



High fidelity (large vocabulary) and low fideli
ty in speech


allow for accents and multiple
human speakers


Visual representation



Very high quality rendering gives a good appearance but does not help the user maintain a good
judgment of space. This seems to depend on very small cues



Stereo vision


s
ize, shape etc



Parallax


Issues

What are the ways the user can interact with space to improve perception?

It would be useful to integrate the above modalities, but there is no standard software.




Evaluation Metrics



We want to do things in Virtual Environme
nts that we can’t do in real life



Trying to measure presence in VE’s makes us realise we don’t really know what it means in the real
world


3



We need a taxonomy, guidelines, corpus, and framework



Dependence on the application domain, for example

-

high fidelity
/high precision

-

general visualisation (e.g. games) where appearance is important,

-

exploratory versus interacting with objects




Software Tools



Existing tools are inflexible, closed and labour intensive



The gap between those who want to study applications an
d the starting point to get there


there is a
high application development overhead



Flexible, open, interoperable, generic tools and interfaces are needed




Design in the light of User Psychology



What is the relationship between technological, physiologica
l, and cognitive elements?



What are the factors that account for individual differences?



When is immersion useful?



There are low level interaction issues and high level learning issues




Large Scale and Small Scale Design


System Integration



Real
-
time modif
ication



Database integration



Supporting the whole design process



Intelligent Level
-
of
-
Detail control



Status control



Common components/file formats


Human Interaction



Human in the loop



Manipulation from within



Multi
-
user manipulation



Multi
-
sire cooperation



Easy exploration and manipulation


Behaviour



Simulation of physical forms


Evaluation



Design approval



Testing




Real World Integration

An example application domain is high precision surgery


Human Factors Needs



Need for outcome measures


developed from re
al tasks with real users



Need to validate training systems with skilled end users



Need to identify where the added value occurs


Perceptual Fidelity Issues



Haptics


high frequency, stiffness, temperature


which is most relevent?



Visual


individual diffe
rences in cue use (shadows, texture) and response to cue conflicts



The visual/haptic interaction


Larger Scale Simulation

There are a number of perceptual issues and questions



4



Trade
-
off between visual information and between vision and non
-
visual informa
tion



Difference between effects on appearance and behaviour



What perceptual information is essential to support and effective VR environment?




Abstract Visualization



Flexible representations



Mapping from abstract to spatial (what works for what task?)



Anch
oring (navigation)



Location


where am I (and where have I come from?)



Evaluation



2. Generating a Research Agenda

A recent joint European Commission/National Science Foundation Workshop was organised to bring
together leading researchers from Europe, USA

and Japan. The overall theme was
“Virtual

Environments
and Human
-
Computer Interaction”.


This joint Workshop was set up under the auspices of the Joint European Commission/National Science
Foundation Strategy Group that had its first meeting in Budapest,

3
-
4 September 1998. The meeting derived
from a joint collaboration agreement between the EC and NSF in August 1998, signed by Dr George
Metakides (Director, Information Technologies, EC) and Prof. Juris Hartmanis (Director, CISE, NSF). The
collaboration

aims to facilitate the joint development of knowledge and applications in key emerging science
and technology areas of mutual interest. Successful cooperation holds the promise of more cost
-
effective
investment of research funds in the USA and the Europe
an Union.


National initiatives in the USA and many European countries are recognizing the benefits to scientific
research in supporting larger groupings, often with interdisciplinary teams of researchers. It is possible to
achieve results with a national

grouping that it is not possible to achieve on the same time scale with an
institutional one. This model has also been used for a number of years by the European Commission to
facilitate research development in European countries, and accomplish faster t
echnology transfer to
European industry by company participation in projects. These initiatives have recently been extended to
include non
-
European partners on a self
-
funded basis. Collaborative links have also been established with
Japan. It is clear t
hat with the increasing globalization of research and development there is a need for
companies to develop products that are viable in world markets. Thus what is being proposed by the EC and
NSF is a logical extension of existing paradigms for securing s
ignificant progress in key research areas.


It was felt desirable to arrange a series of research Workshops to enable early identification of key research
challenges and opportunities in information technology. It was intended that each Workshop should br
ing
together eminent scientists and technologists in the US and Europe in the area being addressed, and that the
themes would emanate from the research community.


At the meeting of the Joint EC/NSF Strategy Group on 3
-
4 September, a number of possible the
mes were
identified. These included "human
-
centered computing and virtual environments", "large
-
scale scientific
databases", and "intelligent implants". Scientists on this Strategy Group included: Prof. Andy van Dam
(Brown University, USA), Prof. Paul Me
ssina (California Institute of Technology, USA), Prof. Rae
Earnshaw (University of Bradford, UK), Prof. Giorgio Baccarani (University of Bologna, Italy), Prof. Rolf
Eckmiller (University of Bonn, German) and Prof. Gilles Kahn (Inria, France).


It was agree
d that the first joint research Workshop should concentrate on the themes of human
-
centered
computing and virtual environments. Human
-
centered computing is perceived as an area of strategic
importance because of the move towards greater decentralization a
nd decomposition in the location and
provision of computation. The area of virtual environments is an area where increased collaboration should
facilitate more rapid progress in solving some of the more intractable problems in building effective
applicati
ons. It is intended that further Workshops should follow this one, either on separate topics or on
specific issues arising out of this first Workshop.



5

The objective of the Workshop was to concentrate on the research frontiers of human computer interaction

and virtual environments. Of particular relevance are the desires that interaction be more centered around
human needs and capabilities, and that the human environment be considered in virtual environments and in
other contextual information processing a
ctivities. The overall goal is to make users more effective in their
information or communication tasks by reducing learning times, speeding performance, lowering error rates,
facilitating retention and increasing subjective satisfaction. Improved design
s can dramatically increase
effectiveness for users who range from novices to experts and who are in diverse cultures with varying
educational backgrounds. Their lives could be made more satisfying, their work safer, their learning easier
and their health

better. Research areas to be addressed included:


-

high level content descriptions and their access, such as metadata and MPEG7

-

reducing cognitive load and providing more scope for creativity

-

cross
-
disciplinary interaction and how to make it work

-

handling

interaction in specific social contexts and with cultural differences

-

dealing with universality and the problems of the differently
-
abled

-

interaction styles and their implications

-

consistency of cognition models across information appliances

-

paradigms for

emerging new kinds of interaction; beyond WIMP interfaces: multimodal and perceptual
user interfaces

-

challenges for virtual environment technology and interfaces

-

usability issues and measuring the effectiveness of symbiosis

-

design and evaluation of online

communities for intranet and internet

-

scaling online communities to support millions of people

-

universal access, social and ethical issues


Detailed results are at
http://www.eimc.brad.ac.uk/news/sch
ools2/



3. Diversity of Technology and Users

The current diversity of the field, such as in displays, is both a challenge and an opportunity. Current
developments in technology and content generation, and the rapid rise of new uses and applications, req
uire
diverse kinds of interdisciplinary expertise in order to exploit the technology effectively. There is a diversity
of technology (hardware, software, and networking), a diversity of users (especially in areas where
technology has not yet made signific
ant inroads) and an increasing gap between what users know and what
they need to know to use current systems effectively. All this bears testimony to tools and systems being
technology
-
driven rather than user
-
driven. Much more attention needs to be given

to end
-
to
-
end design and
integrating the needs of the user from the very beginning. Critical parameters in the design and evaluation
process need to be much more firmly identified, quantified and rigorously upheld. Research is needed in this
area.



4.

Research Integration

The breadth of the field is not being taken into account by current research. The experts in converging areas
are not working together, and research programs are not getting the right kind of interdisciplinary expertise,
or support,
to give added
-
value integration. Indeed, the need for a greater degree of integration and greater
attention to scalability pervade many of the research issues.



5. Multiple Disciplines

One important area of future work is the behavior of individuals and
communities in their relationship to
each other and to the world. There is a long history of educational, psychological, social psychological and
sociological studies, but methodological innovations are needed to capture and understand the complex
nature
of individual and group behaviors that occur while using technology. Analytic and descriptive studies
can provide useful insights, but there is a strong need for more prescriptive outcomes that can guide
designers of new technologies. Guidelines are avail
able for basic user interface design, and these need to be
extended to accommodate new technologies. In addition, validated metrics, user surveys, task taxonomies,
ethnographic methods of observation, participatory design methods, usability testing strate
gies, expert review

6

techniques and software development methodologies would all help produce more orderly development
processes for new technologies. Social impact statements prepared in advance of implementations could
facilitate broad discussions of cri
tical technologies and thereby minimize the number and severity of
unanticipated side effects.


Understanding community relationships becomes even more critical when it is a community of users
interacting in a shared world or information space, such as on
the World Wide Web. The interface needs to
be appropriate to the task to be performed, the social behavior of the user (or groups of users), and the
maintenance of relationships. Research programs should be developed in this area.


Strong encouragement s
hould be given for universities to support multidisciplinary activities and to reform
traditional computer science departments so that they include a human
-
centered approach throughout their
research and educational programs. A specific suggestion for mov
ing the center of gravity in this direction
would be to fund graduate fellowships in human
-
centered systems.


Much stronger promotion of evaluation and empirical testing of systems in the context of work is needed.
This evaluation and testing should be bo
th controlled and ethnographic, in the laboratory and in the field.
Needs, requirements and behavior of the users, as well as the range of problems they need to solve, should be
considered. Progress in VR, online communities, universal usability and othe
r user
-
centered areas will be
dramatically increased if the funders insist on some form of assessment.




6 Key Application Drivers

Problems to be solved can be key drivers in the domain. These represent in some sense "pull" requirements
from the user tha
t need to be considered alongside the more normal "push" technology from the vendors.
One possible agenda for collaboration between industry, academia, and government, is to use the "Content"
age as the key driver for 2010. Content is needed for human me
dia technology, augmented reality, digital
story telling, interactive broadcasting and multimedia workspaces. Indeed, the whole nature of the human
-
computer interface may move away from one operating on a model of sequential task definition and
processing
. The new human
-
computer interface could operate on a model of behavior, context, cultural
background, information awareness and imagination, namely "story
-
telling" at the interface, thus drawing on
its own values of context and history.


A working model
and methodology is needed for the Content Age. The technology should be user
-
centered
and mobile with new types of interaction technology and information display. Three key application domains
have requirements for this technology:


• Health and continuin
g medical education (both doctors and patients)

• Environment

• Cultural heritage


These involve digital content, but more importantly the user perceptualisation of this content and interaction
with it.



7. A Taxonomy of Human
-
Centered Systems

An initial
taxonomy for human
-
centered computing in the context of virtual environments is proposed. This
provides a framework for an understanding of multi
-
channel input and output, the skills of the user, the
particular technology selected for a task and the task
to be performed (whether simple or complex). A
foreground and background task model is proposed, and it highlights two key issues for the future:


• How to get foreground and background to assist each other

• How to increase the effective contribution of
the background (i.e. to make the computer more aware of the
user's context, needs, and requirements at any point in time).



Bibliography


7


This is a short list of useful references and background reading. It is not intended to be an exhaustive
bibliograph
y.


An NSF Workshop produced a special issue of Behaviour & Information Technology Vol 12, No. 2, March
-
April 1993.


A useful Canadian policy document that thoughtfully deals with Universal Access

http://www.fis.utoronto.ca/research/iprp/ua/


A SIGGRAPH agenda document that deals with graphics and user interfaces

http://www.siggraph.org/othercom/whitepaperGII.html

http://www.siggraph.org/pub
-
policy/


"Virtual Reality: Scientific and Technological Challenges",
National Research Council Report, 1995.
http://www.nap.edu/readingroom/b
ooks/


An NSF Workshop on
"Human Centred Systems"

February 17
-
19, 1997

This has many useful position papers and recommendations from four working groups:

http://www.ifp.uiuc.edu/nsfhcs/


"More than Screen Dee
p
-

Toward Every
-
Citizen Interfaces to the Nation's Infrastructure",

National
Research Council Report, 1997.

http://www.nap.edu/readingroom/books/screen/index.html


"Modeling and Simulat
ion: Linking Entertainment and Defense"

National Research Council Report, 1997

http://www.nap.edu/readingroom/books/modeling/


"The Unpredictable Certainty: White Papers",

"Internetwork Infrastr
ucture Requirements for Virtual Environments",

pp 110
-
122, National Academy
Press, 1998

http://www.nap.edu/books/0309060362/html/110.html


The following document is a recent publication outli
ning a basic taxonomy of approaches to VR with special
consideration to their affordances to support collaborative work, both same
-
place and remote.

http://www.
siggraph.org/publications/newsletter/v32n4/contributions/buxton2.html


The following two pointers are to articles describing research from Bill Buxton's group dealing with
technology mediated collaborative work at a distance, reactive environments and ubi
comp.

http://www.dgp.utoronto.ca/people/BillBuxton/index.html#anchor1433082

http://www.
dgp.utoronto.ca/people/BillBuxton/index.html#anchor1447272


Electronic Visualization Laboratory:

http://www.evl.uic.edu


CAVE Research Network Users Society

http://
www.ncsa.uiuc.edu/VR/cavernus/


"Funding a Revolution: Government Support for Computing Research"

Chapter 10,
"Virtual Reality comes of Age",

pp 226
-
249, National Academy Press, 1999.
http:/
/www.nap.edu/books/0309062780/html/226.html


A recent and influential US Presidential Information Technology Advisory Council Report, PITAC report,
http://www.ccic.gov/ac/report



8

Vice
-
President Gore Announces C
linton Administration's $366 Million IT

Initiative

http://access.ncsa.uiuc.edu/Headlines/990209.ITinitiative.html


"What Exactly is the Grid?"

http://access.ncsa.uiuc.edu/CoverStories/WhatisGrid/


ACM CHI99, Workshop 8,
"Development of an HCI Research Agen
da"

http://zing.ncsl.nist.gov/chi
-
research/

http://turing.acm.org/sigchi/chi99/cp/?show=wrk_08


"Networked Virtual Environments
-

Desig
n and Implementation",

Sandeep Singhal and Michael Zyda, ACM Press, July 1999

Contents at
http://www.npsnet.nps.navy.mil/zyda/NVEBook/Book.html



Author Biography

Rae Earnshaw is Professo
r of Electronic Imaging and Media Communications and Head of the Electronic
Imaging and Media Communications Dept at the University of Bradford, UK. He is also Head of the School
of Informatics. He has been a Visiting Professor at Illinois Institute of T
echnology, Chicago, USA,
Northwestern Polytechnical University, China, George Washington University, Washington DC, USA, and
the Swiss Federal Institute of Technology. He was a Director of the NATO Advanced Study Institutes on
computer graphics and CAD in

the UK in 1985 and in Italy in 1987. He is a member of ACM, IEEE, CGS,
EG, and a Fellow of the British Computer Society. He is an Expert Adviser to the European Commission, a
Reviewer for Framework 5 proposals, and an Evaluator of ESPRIT LTR and ACTS pr
ojects.


Prof Earnshaw has authored and edited 28 books on computer graphics, visualization, multimedia, design,
and virtual reality, and published 128 papers in these areas. He is a member of the Editorial Board of The
Visual Computer, a member of the Ed
itorial Board of IEEE Computer Graphics and Applications, Editor In
Chief of Virtual Reality: Research, Development and Applications, Vice
-
President of the Computer Graphics
Society, and Chair of the British Computer Society Computer Graphics and Displays
Group. He is also a
Director of Advance Visual Communications plc.


20 March 2000