Virtual Worlds Technology Roadmap

yardbellΤεχνίτη Νοημοσύνη και Ρομποτική

14 Νοε 2013 (πριν από 3 χρόνια και 8 μήνες)

88 εμφανίσεις

Virtual Worlds Technology Roadmap


December 2008


Seung Wook Kim, Francisco Imai, Stefan Marti

HCI Research Team, Computer Science Lab

Samsung R&D Center, San Jose

SISA CSL “
Innovate by doing”

VW Technology T
rends
A
nalysis

1.
State of The Art

1.1 Virtual World Clients for Fixed Platform

1.2 Virtual
W
orld Clients

for Mobile Platform

1.3 Virtual World Server Technologies


2.
Catalyst Elements for VW

2.1 3D content

2.2 Natural interaction
for

3
D

2.3 A
rtificial Intelligence in VW


3.
M
ixed Reality as a Killer Application




Backup Slides


Mixed Reality Workgroup Mission and Goals


Issues in 3D



-

1

-

SISA CSL “
Innovate by doing”

-

2

-

Trends:



1.
Variety

of VW clients:


-

standalone, Web
-
based (ActiveX or Flash), CE
-
based (game console), etc


2.
Multimodal

in
-
world communication (e.g. textual + bubble + voice chat)


3. Support of user created 3D content via
standard forma
t (e.g. COLLADA)



Challenges:


Insufficient client performance: 3D rendering quality, physics, animation


Voice chat is becoming standard, but it still doesn’t benefit from non
-
verbal communication (i.e., non
-
intuitive avatar gesture/action control)


Complicated 3D content creation pipeline: CAD/CAM skills are required

1.1 State of The Art: VW Clients for Fixed Platforms

VW Web Plug
-
In

VW on CE Devices

Sony’s Playstation Home

VIVATY Scene

for AIM

Opportunities:



Actively apply
multicore/parallel
computing technologies
(for both
CPU and GPU) on VW client
development



Sensor technologies
that will
accurately translate physical motions
into virtual actions



Design
intuitive and intelligent UI
for easier creation and/or
manipulation of virtual objects

Flash
-
based VW

WebFlock SmallWorlds

Standards for 3D content

Forterra’s Olive VW
Platform supporting
COLLADA*

Current status:



The Electric Sheep Company:
Providing Flash
-
based & private
-
labeled VW service,
WebFlock


SMALLWORLDS:

Flash
-
based VW


VIVATY
: Combining Web plug
-
in
Virtual Community (
VIVATY Scene
)
with social networking services
(AIM and Facebook)


SONY
: Providing VW community
service (
HOME
) based on the game
console Playstation3


FORTERRA:

Support for COLLADA*
into its open and standards
-
based
VW platform,
OLIVE

SISA CSL “
Innovate by doing”

Trends:



1. Mobile 3D clients start providing users with
permanent access
to VWs


2. Connecting VW users in real
-
time from
instant messaging client


3. Emergence of
motion sensors
and
haptic feedback
on mobile devices


Challenges:



Limited performance of remote rendering & streaming technologies



No voice/text enabled thin client connecting VW on mobile platform



Lack of new use models for sensor
-
mounted mobile devices



1.2 State of The Art: VW Clients for Mobile Platforms

State
-
of
-
Art Mobile Device for 3D Graphics

Mobile VW client

Vollee’s SL client

Vivox’s SLim client

Opportunities:



Enhance the
performance

of 3D
graphic accelerating processor for
mobile devices



Integrate
VoIP/messaging system
with mobile immersive 3D client



Develop
highly
-
efficient mobile VW
client and its UI mechanism



3D Motion Tracking

Apple’s iPhone with ARM1176JZF & Samsung S3C6400

Thin VW client

-

3

-

Current status:



VOLLEE
: Leader in mobile VW
service with remote rendering &
streaming technologies



VIVOX (w/ Linden Lab)
: First
commercial voice
-
driven IM client
(SLim) for Second Life



APPLE
: iPhone is equipped with a
top
-
level chip (ARM1176JZF) with
3D accelerating coprocessor
(Samsung S3C6400)

SISA CSL “
Innovate by doing”

Trends:



1. Open
-
source approach to
interoperable

server platform


3. Efforts for building
MMOG

environments in VW


2. Both open and
private
-
labeled
virtual world (VW) services are increasing


Challenges:



Client
-
neutral, flexible and highly
-
trusted server platform is required



Difficult to attract and hold a large volume of concurrent users



Server maintenance cost is still expensive (per user)



1.3 State of The Art: VW Server Technologies

MMOG* in VW

Multiverse

Opportunities:



Develop
software

standards

for
interoperable platforms for online 3D
applications and virtual worlds



Connect different
hardware
platforms

(e.g. mobile devices and
fixed systems) via VW applications



Rethink VW
platform architecture

(e.g. cloud computing)


Interoperable VW

Open
-
Source Community

Open Grid by IBM+Linden Lab

OpenSimulator

-

4

-

Current status:



OpenSIM:
Open
-
sourced and

flexible 3D application server
supporting various VW clients



IBM + Linden Lab
: Releasing the
first interoperable VW service
platform (
Open Grid

Beta)



Multiverse
: Providing open
development toolkit for MMOG
-
type VW, including authoring tools
and source code

SISA CSL “
Innovate by doing”

Trends:



1. Renaissance of
3D movies


-

11 movie titles released (from 2005 to 2008) and many more in 2009


2.
Consistent popularity of
games

with 3D content


3.
Explosive increase in
medical imaging
with 3D content ($1.4B by 2013)


4.
Web
-
based VW and Mirror World (SL, Google Earth)


5. Stereo Synthesis


-

2D to 3D conversion
, depth capturing, multi
-
aperture imaging


Challenges:


Displays
: auto
-
stereoscopic displays are expensive and cause user fatigue
and most of 3D viewing solutions require wearing awkward glasses


UX
: there is no good solution in the market for natural and intuitive user
interface for 3D content

2.1 Catalyst Elements: 3D Content

Opportunities:



Create
innovative

and
intuitive
user interaction model for immersive
3D VW client
on consumer electronic
devices



Provide
mobile solutions
for
3D
content navigation



3D interactive movies
for VW



Remote medical service and
training
in VW

-

5

-

Current status:



Philips:
Developed auto
-
stereoscopic displays. Technical
issues such as narrow angle of
view and user fatigue. No
compelling UX


3DV Systems
: Detection of body
gestures using depth cameras for
avatar control. Issues of user
fatigue and lack of resolution


GestureTek:
Control of digital
content by tracking gestures.
Could lead to user fatigue

Source: Insight Media

Source: 3D SMPTE standard activities

Wendy Aylsworth (Warner Bros.)

SISA CSL “
Innovate by doing”

Trends:



1. Beyond keyboard and mouse: evolution towards
alternative HCI
tools


2.
Multi
-
touch:

Evolution towards gesture interfaces
on surface


3. Multitude of
sensors
: Progressive adoption of orientation, position and
motion sensing (optical sensors, inclinometers, accelerometers, compass)


Challenges:


User fatigue
: solutions that rely on mapping human body movements often
lead to user fatigue


Lack of feedback:
it is difficult to give appropriate feedback for 3D
interaction

2.2 Catalyst Elements: Natural UI for 3D

Opportunities:



Combine different technologies to
provide a more
immersive

and
engaging

UX for 3D interaction



Create
closed
-
loop systems
for user
experience and 3D object rendering
by dynamically displaying 3D object
according to user position and
interaction with virtual content

Orange

Seeingmachines

Softkinetic

Mgestyk

-

6

-

Current status:



Seeingmachines:
Developer of
face API that tracks human face
with normal webcam and displays
images based on view
-
dependent
-
rendering


Softkinetic:
Provides 3D real
-
time
gesture recognition middleware


Mgestyk:
Combines 3DV systems
depth camera with hand
-
gesture
processing.

SISA CSL “
Innovate by doing”

Trends:



1. A.I. engines programmed to
learn by interacting with humans
in VW


-

VW gives advantage of embodiment to A.I. engine without hassle or


costs that comes with physical robotics


-

VW allows exponential increase in interaction for AI engines to learn


2. There are efforts to make
VW presence and interaction more engaging


Challenges:


Recreate self
-
mind in VW:
there is need to create a self
-
awareness as
present in human interaction by appropriately planning


Pattern mining
: finding complex patterns requires prohibitively inefficient
searches


Self
-
modeling:
creating learning algorithms capable to embody intelligence


Language and gesture processing:
interpretation of language, gestures

2.3 Catalyst Elements: Artificial Intelligence in VW

Opportunities:



Virtual pets
can be a killer
application for A.I. in VWs



Adding ability to have pets genuinely
learn and respond to the
environment
will make them more
real to the user and increase the
user/virtual pet bond



Autonomous avatars
for creating
rich virtual environments

-

7

-

Current status:



Novamente IRC: methodology for
teaching:



Imitative learning: instructor shows
by example


Reinforcement learning:


student
repeats movements with feedback


Corrective learning:


instructor
actively corrects student’s
movements

SISA CSL “
Innovate by doing”

Trends:


1. Increase in information overlay (Googlemaps, VWs)

2. See
-
through devices as a platform for mixing VW and RW (iPhone App)


Challenges:


Design of wearable displays
: current solutions are bulky and too geeky


Need for an intelligent agent:
user should not be overwhelmed by profusion
of virtual data but system should have an embedded agent that filters
relevant or preferred information


Lack of precision of location sensors
: current location sensors need to
improve precision in order to identify location for data augmentation


3. Mixed Reality as a Killer Application

Opportunities:



Develop technologies for an
unobtrusive system with natural UX
for mixed reality



Possible applications:

-
Travel planning with shared experience
and enhanced participation

-
Training of first responders

-
Inter
-
reality physics: virtual reality
system coupled with real
-
world one

Virtual Continuum (by Paul Milgram, 1994)

-

8

-

Current status:



BBC R&D:
Have explored the
impact of mixed reality in media:
http://www.bbc.co.uk/rd/projects/virtu
al/mixtv/index.shtml



Nokia Research:
Research on
mobile mixed reality system with
augmentation of real world using
mobile phones.



GaTech
: Augmented Reality in SL

Nokia Research

BBC R&D

SISA CSL “
Innovate by doing”

Backup slides

SISA CSL “
Innovate by doing”

Mixed Reality Workgroup: Mission

We believe that
Mixed Realities
have the
potential to
change the way we interact
with the world in a radical way
.


Under
Mixed Reality
, we refer to the merging
of real and virtual worlds to produce new
environments and visualizations where
physical and digital objects co
-
exist and
interact in real time.


But radical change can only happen if the
technology is
available on a broad basis
.


The more people who can interact with
Mixed Realities
, the clearer the advantages
of such a technology will become.

Therefore, our mission is to improve human interaction with real and
virtual worlds by
encouraging the mass adoption of Mixed Reality
technologies
, and the creation of an industry with very low barriers to
innovation and business success.

SISA CSL “
Innovate by doing”

Mixed Reality Workgroup: Goals

We propose to provide information aimed at
technology providers, investors,
and

business
decision makers
:



Casebook

that documents past efforts, lessons
learned, and best practices



Roadmap
of key technical
requirements

for "pacing
mass usages“



“Grand Challenges”
collection

to inspire innovation
in the industry

We are an
Active Workgroup
that develops
documents which frame the usages, as well as key
technical and business model barriers, to the success
of the Mixed Reality vision.

SISA CSL “
Innovate by doing”

Issues in 3D


3D content creation


-

Quality x Quantity


-

Mastering 3D content for home x cinema


-

3D creation and workflow training and education


3D content storage, transmission and distribution


-

File formats, compression, transmission


-

Coordinate work in standard bodies (DCI, MPEG, IEEE 3D Consortium,


3D4YOU, etc…)



3D Promotion


-

Common naming conventions and vocabulary


-

Develop content for web distribution


-

End
-
user and retailer/reseller education and in
-
store demos



3D Displays


-

3D display quality metrics


ghosting, brightness, etc…


-

3D human factors


eye fatigue, viewing time, viewing angle dependency, etc





SISA CSL “
Innovate by doing”

3D Display Technologies

SISA CSL “
Innovate by doing”

3DTV vs. 3D
-
ready TV

SISA CSL “
Innovate by doing”

Cost

SISA CSL “
Innovate by doing”

PDP display market forecast

Samsung manufactures 3D
-
ready PDP
-
TV. This is the expected PDP forecast in
the USA with 3D TVs dominating the market in the next years.




SISA CSL “
Innovate by doing”

3D LCD display market forecast



It is expected that 3D LCD is going to emerge in the market and grow
exponentially

SISA CSL “
Innovate by doing”

3D and 3D
-
ready display market



SISA CSL “
Innovate by doing”

3D content


movie as a driving force

SISA CSL “
Innovate by doing”

TV


How do you know it is 3D
-
ready?

SISA CSL “
Innovate by doing”

3D
-
ready TV


3D glasses and PC requirements

SISA CSL “
Innovate by doing”

3D
-
ready TV


3D glasses and PC requirements

SISA CSL “
Innovate by doing”

3D
-
ready TV


PC requirements