An analysis on the development of virtual reality based interactive simulations and the hardware used for such solutions

sandpaperleadSoftware and s/w Development

Oct 31, 2013 (3 years and 7 months ago)

62 views

1

of
15























An analysis on the development of virtual reality based

interactive simulations and the hardware used for such solutions


Mr. Aaron Richard Colwill

Head of Research & Development

Terreloc Laboratories

arc@terreloc.com


Design for E
ntertainment Systems

BSc Computing & Games Development

University of Plymouth

2013
-
2014





2

of
15







Table of contents



An analysis on the development of virtual reality based simulations.........................................

1


Table of contents............................................................................................................
...........

2


Introduction........................................................................................................
......................

3


The Virtual Environment Solutions..........................................................................................

4


VBS2 by Bohemia Interactive...............................................................
........
........................

4


Unity3D by Unity Technologies...........................................................................................

4


Head mounted displays and visual orientation tracking..............................................
..............

5


Introducing the eMagin Z800 SVGA....................................................................................

5


Introducing the Oculus Rift developer kit...................................................................
.........

6


Fields of view...............................................................................................................
.........

7


Displays.......................................................................................
.........................................

8


Weight.......................................................................................................................
...........

8


Manufacture specifications...................................
................................................................

8


Visual orientation tracking..................................................................................................
.

9


Weapon tracking and controls.........................
.........................................................................

9



Tracking
.............................................................................................................................
.

9


Controls......................
.........................................................................................................

10


Simulation Processing........................................................................................................
......

11


Conclusion

/ How is this information going to be used in the Terreloc TRI solution ..............

12


References
.............................................................................................................................

1
4










3

of
15







Critical re
view introduction


The focus of this paper is the critical review on the approach on the development of virtual reality based
interactive simulations and the hardware used for such solutions with particular focus on military simulation. Four
topics of anal
ysis will be covered and are summarised as: the virtual environment, head mounted displays with
visual orientation tracking, weapon tracking & controls and simulation processing.


Virtual battle
-
space 2 (VBS2) is a software simulation solution provided by
Bohemia Simulations (also known as
Bohemia Interactive or BIS). The software aims at providing realistic simulation of weather, locomotion, vehicle
physics and ballistics (Bohemia Interactive Simulations, 2013). Used in conjunction with bespoke peripherals

such
as the Oculus Rift head mounted display and a simulated firearm this allows for the creation of an immersive
simulation.


Biagini .M (N.D) research showed that the use of modern HMD's and haptic devices in combination with a
virtual simulator e.g. Vi
rtual Battle Space 2 (VBS2) was sufficient for use in pre
-
deployment stages for NATO
units undergoing training. Various companies such as Quantum3D and Intelligent Decisions have realised the
joint capability of the software solutions provided by VBS2 and
that similar systems with modern hardware can
produce cost effective training for dismounted soldiers in pre
-
deployment, active and non active duty roles.



Figure 1: the DSTS (CG
2
, 2013)


4

of
15




One such solution is the Dismounted Soldier Training System (DST
S) (Figure 1). A product provided by
Intelligent Decisions, Inc. that incorporates the ExpeditionDI (Box 1) immersive training by CG
2

Inc, a
Quantum3D company to provide a similar
environment

as relative to the real world as possible. Thus providing
an im
mersive experience where the user is tasked at operating in a virtual combat
environment

where multiple
threat types (infantry, light vehicles, aerial and armoured vehicles) are presented.
The DSTS can be used for
individual, squad training or large scale
training operations and exercises through wireless networking
(Quantum3D, 2013).




Key technologies in ExpeditionDI:

1)

Integrated 'Thermite' backpack computer from Quantum3D is used for visual display,
graphics and networked simulation with other clien
ts.

2)

Head tracking device and head
-
mounted display provides full 360 degree freedom of
movement used for natural movement and enhanced situational awareness.

3)

Non
-
restrictive weapon movement via un
-
tethered operation away from the body which
enables the part
icipant to use their reflexes without any hindrance from the device.

4)

Weapon fore
-
grip/controller provides the participant with forward,

rearward,

left and right
movement coupled with the
participant’s

rotational axis to enable simulated locomotion.

5)

Posture

sensors provide

the participants current stance state to the simulation, enabling them
to crouch.


Box 1 (above): Key technologies of the ExpeditionDI / DSTS (CG
2

, 2013)



On the topic of immersion, Wiebel
& Wissmath

(2011), describe two forms of immersi
on. Firstly 'presence', this
results in the sensation of being present in a remote location or mediated environment through technical
interfaces. Secondly 'flow', the immersion in the simulation with a gratifying mental state of energised focus, full
invol
vement and achievement.


One of the conclusions of Wiebel
& Wissmath

(2011) research showed that motivation factors played a crucial
role in the level of immersion that the user experienced, this was particularly referring to 'flow' immersion.

In the ins
tance of DSTS
-

a militarised solution that doesn't have to cater to the general population, the
experience is intentionally immersive as it plays a critical part of the soldiers training and could essentially mean
life or death. This leads to the soldier
actively wanting to be fully immersed in the simulation as it could help them
through dangerous situations; unlike the standard consumer where no real threat will ever be presented.



5

of
15




The Virtual Environment Solutions


VBS2

Virtual battle
-
space 2 is th
e second iteration of the VBS engine, a simulation engine derived from the Real
Virtuality engine by Bohemia Interactive, which features dynamic lighting with real
-
time shadows, dynamic
weather and fog, large texture mapping for real
-
world terrain using GI
S data, bone
-
based animations to simulate
locomotion and networking for server
-
client simulations and other simulation engine features. These features all
add to the level of immersion that the participant can feel and are essential for a simulation to be
perceived as real
life for training (Bohemia Interactive Simulations, 2013).


The VBS2 software solution costs $3,190.00 per license which features the engine including art assets and an
object oriented scripting interface between the user and the engine f
or creating custom scenarios. No internal
source code is included in
licensing;

this could potentially be problematic when trying to provide solutions
outside the scope of the technology.


Unity

Unity3D by Unity Technologies is a growing popular game engin
e used by many commercial AAA quality game
studios. Unity is a component
-
based engine meaning that all game objects are derivatives of a singular abstract
class which has a transform. There are two versions of the engine, the 'commercial pro' version which

features
dynamic lighting, dynamic occlusion culling and post rendering effects. There is also the 'free indie' version which
lacks the aforementioned features but is accessible to all developers as it is free for commercial use.


Unity3D is a cross
-
platf
orm solution, meaning it can run on Apple Mac, Linux and Microsoft Windows based
systems. The code base has been created with modularity and abstraction as a core feature. As an operating system
agnostic application, the Unity engine uses various layers of

abstraction for operating system features such as
rendering in OpenGL in non Microsoft Windows based systems and DirectX 11 for Microsoft Windows systems
where DirectX is supported.





The head mounted displays


Introducing the eMagin z800 SVGA

The DST
S uses a HMD for visual simulation, this falls into the 'presence' category of the immersion. The HMD
for the DSTS is a helmet mounted eMagin Z800 SVGA OLED display (Taylor & Barnett, 2013).

The solution
also incorporates the InertiaCube positional and rot
ational tracking by InterSense into the display and tactical
visual computers to process the HMD tracking data to provide a fully immersive training environment that can be
operational in any location (Quantum3D, 2013). The virtual system is worn by the pa
rticipants, see figure 2.

6

of
15



Figure 2: eMagin display on the Dismounted Soldier Training System (DSTS) (Biagini .M N.D)




Introducing the Oculus Rift Developer Kit

The Rift by Oculus VR
Inc

is a HMD with a different approach to commercial virtual reality.
The device uses a
single screen to present the virtual environment to the user, unlike the eMagin device as mentioned above. This
factor allows Oculus to produce the rift at a lower price point due to using a cheaper, higher
-
resolution display. It
includes

development drivers for popular game engines such as Unreal Engine by Unreal Technologies, Source
by Valve Software and Unity by Unity Technologies. The drivers are still in development.









Figure 3: The Oculus Rift Developers Kit (iFixit.com, N.D)




7

of
15


Display Field of view comparison

The eMagin device in the DSTS has a 40 degree field of view and causes a divide in the overall flow of the
simulation due to the lack of the device covering the eyes entirely (see figure 2). The
participant’s

peripheral

vision becomes useless and could possibly cause distraction.


The Rift by Oculus Ltd. uses a different approach, the HMD covers the eyes and peripheral vision is warped via
lenses placed between the
participant’s

eyes and the screen. This gives the user a

field of view (FOV) of 110
Degrees compared to the commonly used 40 degrees that conventional HMD's display (see figure 4) (Oculus VR
2013).



Figure 4: Oculus rift field of view comparison



A paper by Toet (2008) suggest
s that military night vision gog
gles typicall
y have a field of view of 30
-
40
˚
,
indicating that tradi
ti
onal HMD’s maybe adequate for night vision simulations. Wider fields of view would be
required to provide a more realistic day time training simulation. However Toet (2008) suggests that

in virtual
environments larger fields of view can cause a greater sensation of motion sickness. This opposes the marketing of
the
Sensics

piSight (another HMD), whom argue that th
eir larger field of view of 150
˚

avoids tunnel vision and
thus motion sickne
ss. However contradictory to their claim, the 'Emulated Military Equipment' that uses a Sensics
zSight only has a field of view amounting to 60º (Sensics 2013).


The paper by Toet (2008) studies the effect of field of view on locomotion, by obstructing par
ticipants’ field of
view and timing them to complete obstacle courses. This paper concludes that locomotion was not impaired
when participants had horizontal

fields of view greater than 75
˚
. It is worth noting however this experiment did
not use virtual reality headsets and was just a restriction of vision experiment.

8

of
15



Locomotion will b
e important for the participants

using the Terreloc simulation. Assuming that
a field of view
greater than

75
˚

is suff
icient, the benefits of the 150
˚

field of view of the Sensics piSight compared to the Oculus
Rift’s 110
˚

would be unnecessary. This will allow for significantly cheaper production as the base cost for the
Sensics piSight is very high compared t
o the Oculus solution.



Display

The Oculus Rift is designed to avoid motion sickness through a higher resolution high density display and
reduction in motion detection sensor latency, although there are still undetermined causes for motion sickness
(Sykes

.T, 2013)


The Oculus Rift features a single LCD high
-
density display with an output resolution of 1280x800 pixels. The
resolution is typically higher than other HMD's on the market due to the increase in demand for small form factor
high
-
density displays

for mobile devices such as smart
-
phones and tablets. LCD displays are typically larger than
their OLED counterpart and slower in response time (Geffroy .B et al, 2006)
;

this could have initially driven
solutions companies such as Quantum3D to use OLED bas
ed screens at a higher price point on the basis that
OLED displays were superior to LCD's. However LCD displays have improved due to a rapid increase in demand
for mobile devices and are more affordable now at higher resolutions, this coupled with new meth
ods of using a
single screen could potentially reduce costs further.


LCD displays also perform well in terms of power consumption, compared to OLED. The organic material in
OLED displays can also cause differential degradation in the colour definition of
the screens leading to visual
distortion. (Geffroy .B et al, 2006)


For the DSTS HMD there are two OLED displays (one for each eye) which
differ

from the Oculus solution of a
single LCD display. Each of these displays output a maximum resolution of 800x600

pixels. This lower resolution
may have resulted in motion sickness observed by Taylor & Barnett (2013) whom evaluated the effectiveness of a
wearable simulation for the military using the DSTS compared to desktop simulators. This lead to their
conclusion
that the wearable simulator was a far less effective training tool and didn't meet its expectations.


As the Oculus Rift will have a higher resolution on commercial release, the Terreloc product could potentially
challenge this conclusion and provide a sup
erior experience where motion sickness has been addressed.


Weight

Toet (2008) suggests that a larger field of view HMD will typically weigh more. The Oculus Rift weighs 375g,
therefore would be a more comfortable head piece than the conventional average o
f 700g
-

1kg weight of a HMD
with a large field of view.


9

of
15


Manufacturer specifications

Table 1 (below) summarises the manufacture specifications for the head mounted displays as analysed above.


HMD

Oculus Rift

eMagin Z800

Sensics zSight

Sensics piSight

Mi
litary Simulation


Terreloc T.R.I


DSTS


Emulated Equipment


N/A

Resolution (Pixels)


1280x800


800x600


1280x1024


2600x1200

Display


1x LCD


2x OLED


2x OLED


2x OLED

Contrast Ratio



>200:1


~10000:01


>800:1

Field of View


110 degrees


40 degrees


60 degrees


150 degrees

Device Weight


375g


<227g

400g


700g


1kg

Price


$300 (developer)


$1,799.00


<$12,995.00


<$12,995.00

Image








Table 1: manufacture specifi
cations for HMDs



Visual orientation tracking

The Oculus Rift uses the
InterSense

MPU
-
6000 six
-
axis gyroscope (a device to measure positional data based on
the rotational axis of the device) and an accelerometer for visual orientation and motion tracking
of the players
head movements (iFixit.com, N.D). This is newer technology compared to the eMagin tracking device and is
more responsive.




Weapon tracking and controls


The use of additional hardware in the DSTS solution enabled the user to essentially

feel an object in the virtual
environment

in the form of a firearm this falls under the category of 'flow' immersion.


The DSTS incorporates a wireless synthetic environment controller designed on the M4A1 weapon system, the
M4 with M203 grenade launcher
and the Squad Automatic Weapon (SAW) M249 light machine gun which is
patent pending (CG
2

2013).


Tracking

Within the DSTS simulation the user controls their avatar's head movement
-

movement of the simulated
10

of
15


weapon and their posture (standing or kneeling),

through their own natural movements which are tracked
through three individual tri
-
axis motion sensors attached to the participants helmet, weapon and thigh. This uses
the InterCube by InterSense, which incorporates gyroscopes into its design.


The wearab
le system proved problematic during its application with inaccurate calibration and tracking of the
weapon control system. Additionally there was interference with the HMD when holding the weapon system in
the correct firing position (Taylor & Barnett, 201
3).


TrackIR 5

The TrackIR 5 uses an infra
-
red spectrum camera to track three points in its field of view. Although this high
-
speed tracking device is expensive for the task it should provide more than adequate performance in tracking the
TRI

simulated

fir
earm in 3D space with positional and rotational transformations.


The approach is completely different to the DSTS tracking system as no gyroscopes or accelerometers will be used
to determine the location of the simulated firearm. Due to the nature of this

solution there is the possibility that
the camera can become obstructed. This could lead to failure in the application until the tracking points come
into the camera’s field of view again.


Calibration of the TrackIR solution is fairly simple


a key pres
s on the firearm will re
-
calibrate the firearm and
set its current position as its
transform
root

position
, just like it does when tracking

a head which is the devices
intended use
.


Controls

Further movement with the DSTS is controlled by a small joystick

and a series of buttons on the fore
-
grip of the
simulated rifle (Taylor & Barnett, 2013), see figure 5.





Figure 5: Button controls on the simulated M4A1 rifle of the DSTS (Taylor & Barnett, 2013)

11

of
15




Although the DSTS uses a simulated weapon, the button

locations do not allow for realistic weapon handling.
(Taylor & Barnett, 2013) the Terreloc solution will position the control buttons at the rear of the weapon in the
pistol
-
grip to allow manoeuvrability whilst manipulating the safety, charging handle an
d magazine.





Simulation Processing


Costello (1997) raises concerns over users of HMD's becoming so immersed in the virtual reality, that they
become functionally blind to the real world, meaning they could collide with real world objects or the cab
ling of
the virtual reality system itself (static positional systems, unlike the DSTS). The DSTS avoids this by attaching the
simulation processing equipment to the back of the participant, in the form of a small backpack, allowing trainees
to move around
freely (CG
2
, 2013).


In contrast to this, the Terreloc system will be designed to be used while standing and sitting down as there is a
potential market for PC simulation enthusiasts for this type of product, as observed within the flight simulation
commu
nity where consumers buy specialist equipment such as replica flight control sticks like Thrustmaster
HOTAS Warthog flight stick.


The specifications of the Terreloc processing solution are compared in table
2

below.



Central
Processing Unit

Random Access

Memory

Graphics
Processing Unit

Hard Drive
Capacity

Thermite 1300

1.4 GHz Intel
Pentium Processor

1GB

ATI Mobility
Radeon X300

10GB

Terreloc
'Cartridge'

Control Center

3.0 GHz Intel
iSeries Processor

4GB

ATI Radeon
HD5770

500GB


Table
2
: Processing u
nit specifications (Taylor & Barnett, 2013).



The higher quality components of the Terreloc 'Cartrid
ge' Control Center maximise capabilities of

additional
hardware for the solution. The simulation should run at a very high frame rate because of this, furth
er decreasing
the chance of the user experiencing motion sickness. This will all be accomplished whilst immersing the player in
the simulation, making it more difficult for the player to notice anomalies in visual feedback such as frame drop
and screen tea
ring.



12

of
15







Conclusion


How is this information going to be used in the Terreloc TRI system?


The Terreloc firearm simulation is a solution for military personnel and serious gamers that provides the user with
a fully interactive weapons system capab
le of simulating the four main components of weapons handling; weight,
ballistics, orientation and action. The key components of this solution are the Unity3D Game Engine running
TRI Software, the Oculus Rift HMD, the TrackIR 5 infra
-
red motion tracking sy
stem and the TRI 'Relock'
weapon system. The simulation will be running on the Terreloc 'Cartridge' Control Center.


Virtual Environment
-

Unity3D

Advantages

Disadvantages

Low cost

Entire ballistics simulation not modelled

Ease of use

No existing terrain

for simulation

Cross
-
platform

No existing 3D models or art assets

Low graphics processing requirement



The HMD
-

Oculus Rift

Advantages

Disadvantages

Higher
-
resolution

Potential for motion sickness although minimised

Higher field of view

Still in ea
rly stages of development

Low latency tracking

Must be attached to TRI Control Centre

Easy to implement


Low cost


Lightweight


Increased comfort



Weapon Tracking


TrackIR 5

Advantages

Disadvantages

Low latency

High cost

Easy to calibrate

Possibi
lity of visual interference caused by external
lighting.

Small form
-
factor
-

won't interfere with firearm

Must be attached to Oculus Rift

Tried and tested within the simulation community



13

of
15


Weapon Controls


Terreloc TRI Pistol
-
grip

Advantages

Disadvanta
ges

Fast simulation

Adaptation from existing controller

Ability to multi task with environment

Has not been tested

Simultaneous movement and weapon manipulation
of the magazine, safety and charging handle.


Analogue movement allowing for fast paced r
unning
or slow walking.



Simulation Processing


Terreloc 'Cartridge' Control Center

Advantages

Disadvantages

Up to date upgradeable components

Higher cost

Faster processing power

Cabling, restrictive movement


user must be tethered
to the system.

Gr
aphical capabilities are higher to allow for updated
versions of the TRI system.



The advantages of the chosen solutions outweigh the disadvantages and have proved to be the most suitable to

the
Terreloc TRI’s

requirements throug
hout this critique. The m
ost re
-
occurring disadvantage

of the Terreloc
solution

is the user being tethered to the system. The effect on immersion will be minimised by th
e higher
specification hardware

to reduce latency and any possibility of immersion breaking emergent scenarios s
uch as a
low frame rate

or asset pop
-
in as the simulation will most likely be in one place
.

It would be immersion breaking
if the military
personnel

or
player had to move around a

lot
; therefore movement will be limited within this
prototype.














14

of
15


References


Biagini .M et al, (N.D). “Immersive Technology Supporting Individual and Collective Training”, available at
http://ftp.rta.nato.int/public/PubFullText/RTO/MP/STO
-
MP
-
MSG
-
111/MP
-
MSG
-
111
-
10.pdf


Bohemia Interactive Simulations, (2
013) “VBS2” available at: http://products.bisimulations.com [accessed on: 22
-
10
-
13]


CG
2

(2013) ExpeditionDI®, CG
2
, Inc. A Quantum 3D Company,
http://www.cg2.com/Products_ExpeditionDI.html

[accessed on: 20
-
10
-
13]


Costello P. J. (1997) ‘
Health and Safety
Issues associated with Virtual Reality
-

A Review of Current Literature’,
Dept. Human Sciences, Loughborough University.


eMagin (2013) Z800 3DVISOR
http://www.3dvisor.com

[accessed on: 21
-
10
-
13]


Geffroy .B, Roy .P, & Prat .C, (2006), “Review Organic Ligh
t Emitting Diode (OLED) technology: materials,
devices and display technologies”, Polymer International 55:572
-
582 (2006) DOI: 10.1002/pi.1974


iFixit.com (N.D), “Oculus Rift Teardown” available at
http://www.ifixit.com/Teardown/Oculus+Rift+Teardown/13682
[accessed on: 22
-
10
-
13]


Oculus VR (2013). “Oculus Rift


Overview” [video online] Available at
https://www.youtube.com/watch?v=1QnXHe_MIx4 [accessed on: 20
-
10
-
13]


Quantum3D (2013) ExpeditionDI, Quantum3D Technology for Solutions,
http://www.quantum3d.com
/solutions/immersive/immersive_training.html

[accessed on: 20
-
10
-
13]


Quantum3D1 (2011). Dismounted Soldier Training System Overview [video online] Available at
http://www.youtube.com/watch?v=AMyoQhUcPgM [accessed on: 20
-
10
-
13]


Sensics (2013) piSight / zSight http://sensics.com [accessed on: 20
-
10
-
13]


Sykes T. (2013), “OculusVR on Rift: Motion sickness solution and 4K resolution not far away” 20
-
10
-
13,
available at: http://pcgamer.com [accessed on: 22
-
10
-
13]


Taylor G. S. & Barn
ett J. S (2013) ‘Evaluation of Wearable Simulation Interface for Military Training’, Human
Factors, Vol. 55, No. 3, June 2013, pp 672
-
690.

DOI:10.1177/0018720812466892


15

of
15


Toet A. (2008) ‘Effects of Field of View on Human Locomotion’, Head
-

and Helmet
-
Mounte
d Displays XIII:
Design and Applications, edited by Randall W. Brown, Peter L. Marasco, Thomas H. Harding, Sion A. Jennings,
Proc. of SPIE Vol. 6955, 69550H, (2008) ∙ 0277
-
786X/08/$18 ∙ doi: 10.1117/12.771950


Weibel D. & Wissmath B. (2011) ‘Immersion in C
omputer Games: The Role of Spatial; Presence and Flow’,
International Journal of Computer Games Technology, Volume 2011 (2011), Article ID 282345, 14 pages
http://dx.doi.org/10.1155/2011/282345