VIRTUAL ENVIRONMENT ON THE APPLE IPHONE/IPOD TOUCH

minedesertSoftware and s/w Development

Oct 31, 2013 (3 years and 7 months ago)

127 views

9
th
 International Conference on Construction Applications of Virtual Reality  Nov 5­6, 2009 
 
VIRTUAL ENVIRONMENT ON THE APPLE IPHONE/IPOD TOUCH

Jason S. Breland
School of Construction, The University of Southern Mississippi
Hattiesburg, MS 39402, USA
jason.s.breland@usm.edu
Mohd Fairuz Shiratuddin, Ph.D.
School of Construction, The University of Southern Mississippi
Hattiesburg, MS 39402, USA
mohd.shiratuddin@usm.edu
ABSTRACT: 3D architecture and construction visualization applications are mostly deployed on desktop, laptop,
and tablet computing devices. Very few 3D visualization applications are developed for mobile devices due to
limitations in computing power and memory. Although there are portable handheld gaming devices that use some
form a 3D game engine to handle 3D graphics quite remarkably, they were never intended for serious 3D
application development.
Recently, mobile devices that can act as a phone or wireless device, a PDA as well as an entertainment system have
been introduced into the consumer market around the world, such as the Apple iPhone, and the Apple iPod Touch.
The rapid change in technology and its affordability has migrated 3D visualization application into these small
ubiquitous mobile devices. Their ability to connect to the Internet wirelessly through Wi-Fi, and/or Third
Generation (3G) network connections, allows for sharing and transferring of digital information among users
regardless of location, at variable speeds. These new mobile devices are also capable of executing 3D applications,
for entertainment and real-time visualization purposes.
Using the Unity game engine for the iPhone/iPod Touch, the authors developed a real-time 3D walkthrough for a
virtual environment application for iPod Touch. In this paper, the authors describe the development process, the two
unique user interaction techniques only available on the iPhone/iPod Touch, the benefits and the challenges, and
finally the potential future improvements to be developed for use in the construction industry.
KEYWORDS: Game engine, mobile, iPhone, iPod Touch, virtual environment
1. INTRODUCTION

Two decades ago, the Newton was developed and introduced by Apple Computers as the first personal digital
assistant (PDA) device. The Newton and other PDAs in the market were designed for personal use and were not
networked for communication purposes. However in recent years, advancements in the communication, Internet and
wireless technologies have led to the emanation of smart-mobile devices that combine features of PDAs, mobile
phones, and handheld game consoles. The iPhone/iPod Touch is one of the most recent smart-mobile devices
introduced by Apple Inc. One main advantage of the iPhone/iPod Touch and similar device is mobility, where users
can carry them anywhere. The ability to connect to wireless networks (3G or Wi-Fi) using the iPhone/iPod Touch
allows for ubiquitous voice and data communication.
The computing power of the iPhone/iPod Touch not only can handle voice and data communications, it can also
handle 2D and 3D handheld quality console games just like the ones running on the Sony Playstation Portable and
Nintendo DS/DSi. Having such computing power opens up new possibilities for the iPhone/iPod Touch to be used
as an architecture and construction 3D visualization tool for teaching and learning purposes as well as in the field.
Devices like the iPhone are now becoming viable tools for offline learning through storage of PDF files for later use
or review (Moore, 2009). More and more portable devices like the iPhone/iPod Touch are being assimilated into
work environments. This paper outlines the development process of a 3D real-time walkthrough in a virtual
environment (VE) application on the iPhone/iPod Touch using the Unity game engine. The authors also describe the
development process, the two unique user interaction techniques only available on the iPhone/iPod Touch, the
9
th
 International Conference on Construction Applications of Virtual Reality  Nov 5­6, 2009 
 
benefits and the challenges of development, and finally the potential future improvements that can be developed for
use in the construction industry.
2. IPHONE AND IPOD TOUCH

The iPhone and the iPod Touch were designed, developed, and marketed by Apple Inc. It is the first mobile device
that supports multi-touch and tilt-control interaction techniques. The iPhone and iPod Touch have similar
functionalities, except the latter does not function as a phone, and does not have built-in camera or microphone. In
Table 1, the authors summarize the comparisons between the iPhone and iPod Touch.

2.1 Specifications of the Iphone/Ipod Touch
 
TABLE 1: A comparison between the iPhone and iPod Touch (Apple, 2009a & 2009b)
 

iPhone

iPod Touch

Dimensions &
Weight

Height:
4.5
inches (115.5 mm)

Width:
2.4
inches (62.1 mm)

Depth:
0.48
inch (12.3 mm)

Weight:
4.8
ounces (135 grams)

Height: 4.3 inches (110 mm)

Width: 2.4 inches (61.8 mm)

D
epth: 0.33 inch (8.5 mm)

Weight: 4.05 ounces (115 grams)

Display

3.5
-
inch (diagonal) widescreen
Multi
-
Touch display with a 480
-
by
-
320
-
pixel resolution at 163
pixels per inch

3.5
-
inch (diagonal) widescreen Multi
-
Touch
display with a 480
-
by
-
320
-
pixel resolu
tion at
163 pixels per inch

Storage Capacity

16GB or 32GB flash drive

Holds up to 20 hours, or 40 hours
of video

8GB, 16GB, or 32GB flash drive

Holds up to 10 hours, 20 hours, or 40 hours of
video

Operating System

iPhone OS

iPhone OS

Battery life

Intern
et use:

Up to 5 hours on 3G

Up to 9 hours on Wi
-
Fi

Video playback time of up to 10
hours when fully charged

Audio playback time of up to 30
hours when fully charged

Internet use:

Up to 9 hours on Wi
-
Fi


Music playback time of up to 36 hours when
fully char
ged

Video playback time of up to 6 hours when
fully charged

Camera, photos,
video recording

Yes

Not Available

Microphone

Yes

Not Available

Network
Connectivity

UMTS/HSDPA (850, 1900, 2100
MHz)
,
GSM/EDGE (850, 900,
1800, 1900 MHz)
,
Wi
-
Fi
(802.11b/g)
,
Blu
etooth 2.1 + EDR


Wi
-
Fi
(802.11b/g)


The iPhone operating system (OS) is relatively simple compared to the Mac OS X that runs on most Apple
computers. The iPhone OS takes up less than 240 Megabytes of the overall storage area of the device. This is
considered small compared to the complexity and functionality of the device. One disadvantage of the OS is its
inability to run multiple applications at once. For example, a user cannot run an instant messaging application in the
background while surfing the Internet. Another drawback is the battery life per full charge. Processing intensive
application such as 3D games, continuous data and voice communication to the wireless network, or playing videos,
reduce the battery life per full charge.
Apple recently releases the iPhone Software Development Kit (SDK) to encourage development of applications and
games from the end-users. This has encouraged end-users to create many innovative and useful applications and
9
th
 International Conference on Construction Applications of Virtual Reality  Nov 5­6, 2009 
 
games for the iPhone/iPod Touch. Applications and games that are approved by Apple are marketed and sold on the
Apple’s “Apps Store” to other users worldwide.
2.2 Limitations of the Iphone/Ipod Touch

According to industrial reviews, the iPhone/iPod Touch lacks some of the features which were expected such as
concerning the touch screen. The size of the touch screen is small that it becomes difficult to click on with great
accuracy. This perhaps becomes most apparent when the user first tries to use the default keyboard which appears
any time texts must be entered. Some users may find that entering little text such as a single sentence is now an
arduous task. In addition, the screen will become smeared with fingerprints. Other limitation include, the camera not
being able to zoom or record video. There is no simple battery replacement to extend battery life, or in case of
malfunction. Another limitation is the network. iPhone users must use the network that Apple has an agreement with
in the user’s country. If users do have the data plan, all internet resources are tethered to the device thus not
accessible by a laptop or other device (Block, 2008).
Another drawback the authors have found is the inability of the iPhone/iPod Touch to run more than one application
at a time. It can be assumed that Apple most likely opted to avoid application multitasking, or tasks running in the
background, as a way to preserve battery life and to avoid overtaxing the limited processing capabilities. With one
or more active program(s) running in the background, the battery could easily drained in a few hours. The exact
processing power of the iPhone and iPod Touch is absent from any known Apple posted information regarding the
devices. In fact users had to create applications specifically designed to tax and record resources just to find out the
exact processing power of the devices at runtime. Snell (2008) found:
“The iPhone, iPhone 3G, and original iPod Touch all report a 400 MHz CPU and 100 MHz bus. The new
iPod Touch reports a 532 MHz CPU and 133 MHz bus. The physical memory remains the same across all
hardware, at about 117 MB (which Hockenberry interprets, I think correctly, as meaning 128 MB of RAM
with 11 MB being used for video).” (Snell, 2008)
3. GAME ENGINES FOR THE IPHONE/ITOUCH

There are several game engines currently available for the iPhone/iPod Touch. Some are specific only for
developing 2D or 3D games, and some support both 2D and 3D games. When Apple first released the iPhone SDK
to the public, the only way custom applications or games can be developed was to use the Objective-C programming
language in the XCode development environment. Objective-C is the native language for the iPhone/iPod Touch
SDK. Developing an application or game using Objective-C can be overwhelming for new developers as all the
elements e.g. the Graphical User (GUI), the game-mechanics, physics, lighting, terrain, etc. of the game must be
created afresh.
However, with the release of the SDK and large interest from the development community, new game engines for
the iPhone/iPod Touch were developed for commercial use and others as freely distributed as Open Source. Table 2
shows some examples of the currently available game engines for development of 3D games on the iPhone/iPod
Touch. Unlike game engine for the PC, the number of 3D game engines available for the iPhone/iPod Touch is still
limited.
TABLE 2: Examples of 3D game engines for the iPhone/iPod Touch
Game Engine

Developer

Status

Website Link

Sio2Engine


Sio2
Interactive

Open Source

http://www.sio2interactive.com/HOME/HOME.html

Oolong


Canno
t be
determined

Open Source

http://www.oolongengine.com

Raydium 3D

Cannot be
determined

Open Source

http://raydium.org/data.php

Ston3D for
iPhone


StoneTrip

Commercial

http://www.stonetrip.com/shiva/publish
-
3d
-
game
-
on
-
iphone.html

9
th
 International Conference on Construction Applications of Virtual Reality  Nov 5­6, 2009 
 
Unity iPhone

Unity
Technologies
Commercial
http://unity3d.com/unity/features/iphone-publishing
Bork3D

Bork3D
Games
Commercial http://bork3d.com
Torque 3D for
the iPhone

GarageGames
Commercial
http://www.garagegames.com/products/tge/iphone
 
3.1 The Unity game engine

Having tested a few of the available game engine for the iPhone/iPod Touch, the authors selected the Unity iPhone
Advanced game engine because of its features, ease of use, faster rate of development and deployment of
applications, and its file format support. The Unity game engine is developed by Unity Technologies. Initially, the
Unity game engine was only available on the Mac OS X platform for games development with options to deploy the
games online, on a Windows PC, or on a Mac OS platform. However, recently Unity Technologies releases Unity
2.5 that allows for development on both Max OS X and Windows platforms. The Unity game engine supports the
development and deployment of applications on the iPhone/iPod Touch through the Unity iPhone Publishing (either
Basic or Advanced) software.
The Unity and Unity iPhone Advanced development environment is mostly GUI driven. Like most game engines,
pre-defined modules already exist in the Unity engines. Modules can be dragged and dropped into a project so that
they can be used in a scene. At the very basic level, the modules can be used are they are. However, extended
functionalities can be easily added to any of the existing modules through the use of Java scripts or C# programming
language (see figure 1). These modules can also be imported and reused in other future projects. Figure xx shows an
actual module used in this paper that uses C# programming language. This module sets the screen of the
iPhone/iPod Touch to landscape display mode when virtual environment is displayed.

FIG 1: The SetScreen module uses the C# language

4. DEVELOPMENT PROCESS OF A VIRTUAL ENVIRONMENT APPLICATION ON
THE APPLE IPHONE/ITOUCH

Most modern 3D games are played in virtual environments (VEs). The main components of the VE are terrain,
skybox, lights, 3D assets such as buildings, avatars (to represent player and non-player characters), textures such as
the ones applied to the 3D model, terrain, avatars, skybox etc., audio such as sound effects, environmental sound,
background music etc., game mechanics and physics. The VE that is created on the iPod Touch consists of a model
of a Country House placed on a flat terrain. The game mechanics and physics components of the VE were excluded.
Figure 2 shows an overview of the development workflow. Each development stage is described in the following
sub-sections.
9
th
 International Conference on Construction Applications of Virtual Reality  Nov 5­6, 2009 
 

FIG 2: An overview of the development workflow

4.1 Development prerequisites

Development of an iPhone/iPod Touch application (or apps) requires the followings:
1. An Intel-based Mac computer.
2. Mac OS X Leopard operating system software installed.
3. An iPhone or iPod Touch hardware.
4. The iPhone SDK available for free from Apple’s website.
5. The Xcode software development tool. Xcode is a suite of tools for developing software on the Mac OS X.
Xcode can be downloaded free of charge from Apple’s website.
6. Join the iPhone Developer Program, and be a licensed Apple developer. A standard program will cost
$99/year. However, by joining the iPhone Developer University Program, Apple will grant free developer’s
license to any academic institution that will teach iPhone application development as or part of a course.
(Note: Apple recently updated the iPhone OS to version 3.0 in June 2009. Following the new OS release, the iPhone
SDK was also updated to version 3.0. Prior to using the new iPhone SDK, the Mac OS X operating system must be
upgraded to version 10.5.7, the iTunes software to version 8.2, and Xcode to version 3.1.3.)
Once the authors became Apple certified developers, access to the iPhone Developer Program portal website was
granted. Using this portal, a Provisioning Profile and an iPhone Development Certificate were created and installed
onto the iPod Touch, to allow for deployment of new locally developed applications on to it. Since the authors are
using the Unity iPhone Advanced game engine, the Unity Remote application was installed first on the iPod Touch.
The Unity Remote application provides a live preview on the iPod Touch screen by directly linking the Unity Editor
environment on the Mac computer to it. This is a useful feature as it allows for quick testing and debugging of the
application under development.
One of the development intentions is to use any 3D CAD model developed elsewhere and did not originate from the
authors. The reasoning behind this is to test out the level of compatibility of the Unity game engine with respect to
3D CAD model. In this paper, a commercial 3D CAD model of a Country House acquired from BlueBrain3D
(2009) was used as part of the VE.
4.2 Terrain

The author separated the development of the 3D CAD model since it was acquired from BlueBrain3D (2009) . The
development of the VE for the iPhone/iPod Touch began with the creation of the terrain. Unlike the Unity game
engine, the Unity iPhone game engine does not have a dedicated terrain engine; hence terrain can only be created
either using geometries created in another 3D modeling software such as 3DS Max or Google SketchUp, and by
Development of VE 
9
th
 International Conference on Construction Applications of Virtual Reality  Nov 5­6, 2009 
 
using a plane geometry (to represent a flat terrain) that is available in the Unity iPhone game engine. In this paper,
since the VE depicts a Country House in a suburban area, the use of a flat terrain was sufficient.
4.3 Skybox

Skies were applied using a skybox method. A skybox is a six-sided cube consists of six-seamless images that
enclose a virtual environment. Each side of the cube represents North, East, West, South, Top and Down
(Shiratuddin et al, 2008, Kanalakis, 2008, Busby et al, 2004). Figure 3 shows the images used for the skybox. A
skybox method is usually employed in computer and video games. This method is very efficient since it will make
the virtual environment looks larger. Skybox can contain skies, mountains, distant objects such as buildings, and any
other objects that are “unreachable” by the user. A skybox can be created using software like Terragen. Another
method of creating skies in a virtual environment is to use a Skydome, which is a similar concept to Skybox, but
uses a sphere or hemisphere instead. The Unity iPhone Advanced engine supports the rendering of Skyboxes
through the use of the built-in RenderFX/Skybox shader.
4.4 Lights and shadows

Lights and shadows play an important role in a virtual environment. Without proper lighting and shadows, a virtual
environment may look plain and dull. Since the Country House model used in this VE application did not have any
interiors, only exterior sunlight was used to generate shadows. A “Directional Light” was used to represent sunlight.
A directional light source illuminates a virtual environment using a constant illumination from a single direction.
Since a directional light source it is usually dynamic in nature, and it uses a real-time stencil shadow casting method
(Shiratuddin et al, 2008). However, in the Unity iPhone Advanced engine, shadows created using directional light
sources do not cast real-time shadows; the shadows were baked onto the textures where lights were directed to,
using shadow mapping techniques. This gives the illusion of shadow casting. The shadow mapping technique
increases frame-rate-per-second (fps) in a VE and prevents the engine from rendering shadows in real-time. This
method may not be suitable in virtual environments that require real-time shadows generation e.g. in studies
involving determination of shadow intensities and shadow projections in real-time.
With the terrain, skybox and sunlight in place, the VE is now partially assembled in the Unity Editor. Since there is
no interaction technique implemented yet, the VE is static.

FIG 3: The skybox images used in the VE
9
th
 International Conference on Construction Applications of Virtual Reality  Nov 5­6, 2009 
 
4.5 Interaction techniques

Interaction technique is one of the crucial elements in a VE. Some VE applications failed due to improper design
and implementation of good interaction techniques. Poorly designed interaction techniques will make it difficult for
users to interact with the VE. The iPhone/iPod Touch has unique interaction techniques. The two main interaction
techniques are multi-touch and tilt-control movements of the iPhone/iPod Touch itself. For example, in the Google
Earth for the iPhone application, by simply tilting the iPhone/iPod Touch up and down, left and right, a user can
navigate through the world map. In this paper, the authors implemented both types of interaction techniques for the
VE to see which of the two is most suitable for navigation purposes.
Revelle and Reardon (2009) reached several important conclusions when designing learning applications for
children in their research. Many of these principles deal with the unique interaction techniques of the iPhone and the
problems some of the children had with the interface. Simplifying the game structure was one way to help users
from being confused. With reduced screen size and new interaction techniques it is best to keep the game play
simple to avoid unnecessary confusion. Using large, visually distinct hotspots is another of their recommendations.
The screen needs to be as simple as possible with large buttons so that they can be easily selected. Too many buttons
or buttons that are too small quickly become difficult to select without error. Tilt functionality can be tricky for
inexperienced users so starting with a low sensitivity is recommended. Users can accidently activate undesired
commands if tilt functionality is too sensitive. A setting to adjust sensitivity for advanced users might be useful. The
use of audio cues or prompts to describe what actions the user should take can also be helpful. Additional simplified
instructions can be applied after 7 to 10 seconds of being idle if the user is still uncertain of what to do. While these
are not all of the principles described by Revelle and Reardon (2009), they are some of the most pertinent to
navigating a 3D world. These should be considered in the early design phases to avoid problems later.
When first developing the VE, the authors implemented multi-touch interaction techniques (see figure 4). Using this
technique, a user simply uses his/her thumbs to navigate in the VE. 2D navigation compass-like images were
overlaid on the iPod Touch screen and used as the areas where a user can touch to initiate navigation in the VE. The
right compass image once touched, enables a user to look around, and the left compass image allowed forward,
backward, left and right movements in the VE. By touching both left and right navigation compass, a user can
comfortably navigate in the VE displayed on the iPod Touch’s screen.

FIG 4: The author using the multi-touch interaction technique to navigate through the VE on the iPod Touch
Next, the authors implemented tilt-control movements. Using this method of interaction technique, a user simply
tilts the iPod Touch in the direction of navigation to move forward and backward, left and right. This technique
however, allows for only navigation in the VE. Looking up and down, left and right could not be combined with this
method of interaction, and this posed a navigation problem on the VE. The authors also discovered that if a user
tilted the iPod Touch too much, large tilting angle made it hard for the user to see the screen. Due to this reason, the
authors concluded that the multi-touch interaction technique is more suitable for navigation in a VE on the
iPhone/iPod Touch. Figure 5 shows a snapshot of Unity Editor loading and at the same time running the VE on the
iPhone simulator application.

9
th
 International Conference on Construction Applications of Virtual Reality  Nov 5­6, 2009 
 




FIG 5: The Unity development environment and the iPhone Simulator
5. BENEFITS, CHALLENGES AND CONCLUSION

Some of the benefits of utilizing a 3D game engine, specifically the Unity iPhone Advanced game engine are two-
fold. Applications and games can be developed fairly rapidly since the Unity game engine comes with basic
modules for real-time 3D applications and games development. These modules are highly customizable through the
use of Java scripts or C#. The Unity game engine also has support for major industry standard file formats ranging
from 2D images to 3D CAD models. This feature alone is an immense advantage since developers no longer have to
be concerned with proprietary non-standard file formats.
The benefits of utilizing the iPhone/iPod Touch for 3D architectural and construction visualization are also clear; the
mobility, the sufficient computing power to run 3D real-time applications, the ubiquitous connectivity to wireless
networks, and the unique interaction techniques pose a new but rewarding challenge in creating a visualization
application. For example with the increased mobility brought on by a handheld device, a client can be briefed about
his building over a lunch meeting. Instead of using a laptop, the client can download, visualized, navigate, and
walkthrough the 3D model of the building. This type of application can also be used as a review tool on the job site.
It could also be used in a small meeting to show the end results of proposed project where a projector or computer
may be unavailable or overly cumbersome.
During development, the authors encountered some challenges primarily due to the hardware and software
limitations of the iPhone/iPod Touch. Screen size and 3D scene complexity are perhaps two of the largest
limitations. The small screen provides excellent picture quality but if the display is cluttered with numerous GUI
elements, it can quickly become difficult to click on or begin taking up screen space. The authors have not
thoroughly tested what is the 3D scene size limit the iPhone/iPod Touch can display in real-time. However, as with
other combination of game engine and hardware specifications, this is an expected limitation.
In conclusion, the authors have succeeded in developing a clean and efficient VE that is easy to navigate and walk
through on the iPhone/iPod Touch. The VE application is functional considering the iPhone/iPod Touch size and
Scene assembler window

iPhone Simulator
Real-time virtual environment simulator
9
th
 International Conference on Construction Applications of Virtual Reality  Nov 5­6, 2009 
 
processing power. The authors’ future investigation is to test the exact limitations of the Unity game engine and the
iPhone/iPod Touch hardware in detail by comparing frames per second with model complexity and different lighting
conditions as well as various navigation techniques unique to the iPhone/iPod Touch. The authors hope to get
further opinions and comments from architects, design reviewers, project developers, and educators etc., who may
find this type of tool useful in a real world scenario. In short, development for the iPod Touch/iPhone is an ongoing
process that the authors’ hope will fully utilize the new interaction techniques and 3D processing power of the
devices which make them so unique for development.
 
6. REFERENCES

Apple (2009a), iPhone – Technical Specifications. Website: http://www.apple.com/iphone/specs.html
 
Apple (2009b), iPod Touch – Technical Specifications. Website: http://www.apple.com/ipodtouch/specs.html
 
Apple Developer Connection (2009), iPhone Developer Program.
Website: http://developer.apple.com/iphone/program/
Block, Ryan (2008), iPhone 3G review. Website: http://www.engadget.com/2008/07/11/iphone-3g-review/
BlueBrain3D (2009), Bluebrain 3D Model Library. Website: http://www.bluebrain3d.com/3d-models/
Busby, J, Parrish, Z. and VanEenwyk, J. (2004). Mastering Unreal Technology: The Art of Level Design
Sams, ISBN-10: 0672326922.
Kanalakis, J. (2008), The Complete Guide To Torque X. AK Peters, Ltd: Wellesley, MA
Moore, J., Oussena, S. and Zhang, P. (2009), A portable document search engine to support off-line mobile learning,
IADIS International Conference Mobile Learning 2009, 26th February 2009, Barcelona, Spain.
Revelle, G. and Reardon, E. (2009), Designing and testing mobile interfaces for children, Proceedings of ACM IDC
2009: The 8
th
International Conference on Interaction Design and Children, pp. 329-332, Como Italy, June
3-5, 2009.
Snell, Jason (2008), That iPod touch runs at 533 MHz.
Website: http://www.macworld.com/article/137139/2008/11/ipod_touch_speed.html
Shiratuddin, M.F, Kitchens, K. and Fletcher, D. (2008). Virtual Architecture: Modeling and Creation of Real-Time
3D Interactive Worlds, Lulu.com, ISBN-10: 1435756428.