Virtual Conference Room

quartercircleAI and Robotics

Nov 14, 2013 (4 years and 1 month ago)

61 views

Virtual Conference Room

December 6, 2001

Group No. 26


Dylan Walker


Team Leader

Anna Alaimo

Sylvester Nowak

Saul Rojas

Suzanna Yau

Mark Shirley

Abstract


Virtual Reality Conferencing


Hardware


Sensors, Receivers, Processors


Digitizal Representation of the body


Software


Client/Server Communication, Client Rendering


Integrates multiple users. Renders perspective to video output


Designed for use with:


Commercial HMD/Datagloves


Continuation: Senior Design Project Opportunity


Inexpensively manufactured wireless HMD/Dataglove


Voice/Textual communication add
-
on to Server/Client package


Multiple application interface: Adobe, Microsoft packages available
within the conference room.

Background


Virtual reality applications include data gloves and head
mounted displays


Position detection is usually an expensive process because of
the hardware used and complicated due to cumbersome and
processor
-
demanding image processing techniques.


Our design provides a cheaper alternative (using IR technology)
feasible for home users with virtual reality dreams.


Position Detection Overview (1)


A user’s position is detected via detection of
individual body parts. An array of infrared LED units
is placed on each body part.


The IRLED units are pulsed in sequence via a power
source and timer chip located on the users body in a
Time Division Multiplexing (TDM) scheme.


The intensity of each IRLED unit pulse is detected by
3 off
-
body phototransistors, allowing for the
determination of the IRLED’s (x,y,z) coordinate via
triangulation.


Position Detection Overview (2)


The output of the 3 phototransistors is fed into 3 pins of an off
-
body micro
-
controller where it is converted to a digital signal by
A/D sampling, demultiplexed into the individual IRLED unit
signals, stored in memory, and used to determine the (x,y,z)
coordinate of the IRLED units.


The output of the micro
-
controller is fed to a computer port for
further processing by the software.



Position Detection Diagram

MCU

PC:


Client Program

Position Sensor Design System (1)


Consists of 3 major subsystems:


STU



Sensor Transmission Unit


Located on the user’s body


Consists of


High power IRLED sensor units


Timing circuitry


Power supply


SRU



Sensor Reception Unit


Located off
-
body


Consists of


3 IR phototransistors at fixed relative locations

Position Sensor Design System (2)


MCU



Micro
-
controller Unit

Located off
-
body

MC68HC12 micro
-
controller


STU


IRLED Units & Rings (1)


Design implications of infrared technology:


IR LED’s are relatively inexpensive.


Transmission is limited to line of sight.


IRLED’s have a specified angle spread, (typically < 90
º)


To account for IRs limitations:


Each
IRLED unit

consists of a hemisphere of IRLED’s to
cover a 180
º solid angle, to minimize the probability of
missing the IR phototransistors.


The IR LEDs in each IRLED unit are pulsed
simultaneously.

STU


IRLED Units & Rings (2)


IRLED units are grouped in
IRLED rings

to provide
redundant information regarding the position and
rotation of each appendage; this minimizes blocked
signal problems.


The IR LED Units in each ring are pulsed in
sequence.

STU


Diagram of IRLED Unit

IRLED





STU


Diagram of IRLED Ring

IRLED Ring

IRLED Unit

STU


Circuit Schematic

1 x 64
demux

E

S
1

S
2

S
3

S
4

S
5

S
0



D
0

D
1

D
63


6
-
bit
Timer/Counter

GND

PWR

C
0

C
1

C
2

C
3

C
4

C
5








LED Unit 0

9V

POT

SRU


Triangulation of IRLED Unit

V
i



voltage drop across phototransistor i


I
i



received intensity of IRLED unit signal at
phototransistor i

D
i



distance from IRLED unit to phototransistor I


Notice: I
i



V
i



D
i


IRLED Unit

IR Phototransistor


z

y


x

D


D


D
3


D
2

D
1

(x,y,z)

(x
3
,y
3
,z
3
)

(x
2
,y
2
,z
2
)

(x
1
,y
1
,z
1
)

Assuming x
1
= y
1
= z
1
= 0 ,
the equations of three
spherical surfaces are:


2
2
2
2
3
2
2
2
2
2
2
2
2
2
1
)
(
)
(
)
(
z
D
y
D
x
D
z
D
y
x
D
z
y
x
D












The solution of this set of equations is:

2
2
2
1
2
2
2
2
1
2
2
3
2
2
2
2
y
x
D
z
D
D
D
D
y
D
D
D
D
x










STU/MCU


TDM Operation


IRLED Rings of various radii are placed on the user’s body
appendages


The IRLED units of each ring are attached to respective signal
line outputs O
i


The following slides shows an example of a single IRLED ring on
the user’s forearm operating in TDM.

STU/MCU


TDM Operation

1 2 2.5 4 5 5.5 7 8 time(

sec)

V
(mV)

Voltage across phototransistor i

STU/MCU


TDM Operation

V
(mV)

1 2 2.5 4 5 5.5 7 8 time(

sec)

Voltage across phototransistor i

STU/MCU


TDM Operation

V
(mV)

1 2 2.5 4 5 5.5 7 8 time(

sec)

Voltage across phototransistor i

STU/MCU


TDM Operation

V
(mV)

1 2 2.5 4 5 5.5 7 8 time(

sec)

Voltage across phototransistor i

STU/MCU


TDM Operation

V
(mV)

1 2 2.5 4 5 5.5 7 8 time(

sec)

Voltage across phototransistor i

STU/MCU


TDM Operation


The output of the three phototransistors are connected to 3 separate
MCU port pins for A/D sampling.


A/D sampling is initiated with each body frame by a short sequence of
synchronization pulses that are not sampled or recorded.



A/D sampling is performed on the pins, with a sampling period T
S

that
is some fraction 1/n of the IRLED unit pulse time (including guard
time).


The maximum voltage of each section of n pulses is selected.


Below, sync pulses are not shown.



1 2 2.5 4 5 5.5 7 8 time(

sec)

V (mV)

MCU
-

Flowchart



[1 x
1

y
1

z
1
]

[k x
1

y
1

z
1
]

Port B


Output

P
3

P
2

P
1

Port D


A/D
Sampling

1
st

n samples

kth n samples




Memory

Microprocessor


(Select k maximums,
one from each set of n
samples)


Memory

Microprocessor


(Obtain k (x,y,z)
coordinates; pass the
data: [i x y z])

To Serial port
of

PC/Client

Software

MCU Role




Render and communicate data from SRU to client software


Process both ordinary and intrinsic transmission signals from
SRU


A/D conversion of SRU data


Detect fallacious signals from the SRU and transmit them to
client software


Calculate scaled coordinates and transmit them to client
software


Able to store and process raw data without any noticeable time
delay (real time)



Flexible and powerful enough for further expansion of the
project

Pin layout of MCU


M68HC16Z1

Pinout of M68HC16Z1, 132
-
Pin Package.

Architecture of M68HC16Z1

Block diagram of M68HC16Z1

A/D Conversion


SRU receives then transmits the raw data (voltages) to MCU


M68HC16Z1 contains a programmable ADC module


number of
automatic conversion modes


A/D conversion only needs four registers and it will performed
using a similar algorithm showed below:

A/D Conversion


Configuring


V
rh

and V
rl

are analog reference voltages and they are connected
directly to analog reference pins


Analog input pins (AN[0:7]) have to be within the range of
specified by V
rh

and V
rl



Use of bypassing capacitors or low pass filtering is
recommended for analog reference voltage and input pins since
any noise into these pins lead to erroneous data

A/D Conversion


Preliminary Code

ADC.ASM

Implementing the MCU



Assemblers


Motorola HC16 Assembler


Compiler


Debugging tools


Evaluation module (EVM)


M68HC16Z1EVM


In
-
circuit emulators


Logic analyzer pods


Debug monitors


Source
-
level debug monitors

MCU Setup


M68HC16Z1 Setup









Network Application Overview




The network application is responsible for successfully obtaining
data from the client hardware, transmitting data between client
and the server application and passing the data over to the
rendering application.


Network Application Diagram

Client PC

Serial Port

Server

Rendering
Application

Network Application Overview


Data is obtained from the PC’s serial port


The client PC runs an application which connects to
the server and exchanges data with the server
application


The server application is capable of accepting and
exchanging data form many user applications


The client application receives data from the server
which is then passed on to the rendering application

Client Application Design


Performs three tasks:


Read Data from Serial Port of the PC


Use system function calls to read the data


Store the data as a program variable


Establish Connection with the Server and Exchange Data


Create a AF_INET domain socket for communication


Send and receive data over the socket


Pass the Received Data to the Rendering Application


Create a AF_UNIX socket for communication with the
rendering application


Write data received from the server to local socket


Server Application Design


Performs two tasks:


Listen for incoming connections


Establish connection with authorized clients


Receive data from connected clients


Exchange data with all connected clients


Create a data bank of data gathered from all
connected clients and server statistics


Create a unique data source for each client from the
database


Send data to each client



Client


Server Application Diagram

Client Application

Server Application

Data Bank

Listening
Server

Rendering
Application

Client Server

Data

Client


Server Connection

Berkeley Socket API will be used to establish client server
communication.


The AF_INET domain socket type will be used with UDP protocol
to establish client


server communication.


UDP (User Datagram Protocol) Protocol


Advantages


Inexpensive in terms of resources


Fast
-

no associated connection setup time


Disadvantages


data is sent but not acknowledged


no guarantee of delivery


Client


Server Application Design

Internet

Client

Server

LAN

Client Server Communication


Client Server Communication utilizes the following types of
messages:


Control messages


Establish a connection



Send server and client statistics


Data messages


Coordinate information from client


Data ready for rendering sent from server to each client


Data Message Format

34

1

00001101

Cell Number

Data Status

Data

-

Cell Number


identifies the data message and is used for
check of data loss

-

Data Status


0 => no change 1=> change

-

Data


coordinate number

Server Data Bank


Store all client data


Each client is represented by a number of sensor units


Each sensor unit has X,Y,Z coordinate information


Server Data Bank Diagram #1

Z

Y

X

Coordinates

Sensor Unit #

1 2 3 4 5 6

Single Client Representation

Server Data Bank Diagram #2

Client #1

Client #2

Client #3

Data for additional clients is stored as a new layer
in the data bank.

The Rendering Application


The Client application must pass data received from server over
to the rendering application.


Create a AF_UNIX domain socket on the local file system


The socket allows for fast and reliable communication
medium


Data is written to the socket

Rendering
Application

Client
Application

AF_UNIX
socket

Design Requirements


Create a virtual reality conferencing package to allow multiple
users separated geographically to participate in a virtual room
conference


Track motion of users using on body sensors



Represent position and motion of each participant at any given
instant in the software environment


Design and build hardware necessary for tracking user body
positioning


Create software necessary to render virtual room and
participants


Create software necessary for client server communication


Create software to operate the MCU


Provide detailed cost analysis of the entire project

Programming a Virtual Conference Room (1)

OpenGL is an open source
graphics API used for 3D
and virtual reality imaging.

Programming a Virtual Conference Room (2)


Create client
-
side rendering scheme using OpenGL


Rendering scheme will be done in first person perspective


Created for use with a commercial Head Mounted Display
(HMD).


Open
-
ended design for future application development


Strategies


Modular design:


All tasks are performed by specific and isolated procedures
to facilitate the future enhancements of this software
package


Data structures: two 3D arrays


One array: data of position for each sensor on the body


Second array: indicates whether or not limb sensor position
changed since last frame or corresponding data was lost


Proposed Solutions to Programming Issues


Redundant data:


Since each limb has multiple sensors to represent the
different sides of it (i.e., underside, top…), lots of redundant
data results.


Proposed Solution:


In many cases, only data from a couple of sensors are
required to accurately determine the position of the limb.


Only this minimum required data is used to render the
person, which will increase efficiency.


It is important to note that redundant data has its
guarantees against lost data in any communication protocol;
at best, there is only acknowledgement of received/lost data


Blocked Sensors



Blocked Sensors, i.e. hand behind back


-

Each limb has multiple sensors

to measure the position.


-

Infrared sensors might be

blocked depending on position

of user.


-

Position of blocked limb may not

be ignored (i.e., must still be

drawn) because another user

should be able to see this limb.


Blocked Sensors
-

Solutions


Proposed Solution:


A blocked sensor will generate zero values for its
position.


A blocked sensor will not necessarily cause a
problem since the multiple sensors per limb result
in many redundancies.


If enough sensors on a limb are blocked then the
limb’s position is not explicitly given


This data must be interpolated with the data
received from adjacent limbs


Lost Packets (1)


Lost packets:


in the UDP communication protocol, we must deal with the
possibility that the packet never arrives.


Since each packet is numbered sequentially, lost packets will
be indicated by an out of sequence packet.


Proposed Solution:


several ways to deal with this; solution chosen based on
which has highest priority.


default solution: use the most recent data received for that
limb.


In the event that the data from the other sensors of the limb
have not been received, data must be extrapolated and
interpolated from adjacent sensors.


Lost Packets (2)


This method presents the more accurate solution, since the
limbs are connected and therefore must move together.


Effects of lost packets


not serious since rate of change is approximately 60 frames
per second.


3D arrays of data are constantly being updated, losing one
piece of data has very little effect on overall animation.




Economics


Prototype model (~$30,000)


Cost estimation


Research and development


Salaries


Direct labor and materials


Indirect materials and miscellaneous expenses


Commercial production (5 year project)


SEEM (Spreadsheet Engineering Economics Model)


Revenues


Expenses


After
-
tax analysis


Break
-
even analysis


Income statement


Capital and depreciation




Commercial Production


Capital & Depreciation


Capital


Working capital = $1,200,000 (Investors)


Borrowed capital = $500,000 @ 4% APR


Depreciable capital = $1,475,000 (Assets)


Total capital = DC + WC = $2,675,000


Current debt ratio = BC/TC = 18.69% (GOOD)


Depreciation


MACRS depreciation


Ending value (Salvage value)

Revenue


Product and services: VCR Kit, Expansion Kit, Technical Support


Revenues


VCR Kit = $17,725,000 @ 116,500 units


Expansion Kit = $4,190,000 @ 46,000 units


Tech Support = $5,000,000 @ 5,000 units


Total Revenue = $26,915,000


Total average units per year = 33,500 units


Total average sales per year = $5,383,000


Expenses


Inflation rate = 2.5%


Direct materials


Variable factory overhead


Total variable costs = $12,763,000


Annual fixed expenses


Selling expenses = $1,000,000


R&D and administrative expenses = $4,739,240


Total fixed costs = $5,739,240 (Average per year = $1,147848)


Total operating expenses = $18,502,240

After
-
tax analysis



FOM (Figures of merit)


Cumulative NPV = $3,774,743


Baseline MARR = 15%


IRR = 89.23%


Capital Gain/Loss


Depreciable assets


Plant
-

$127,500


Production equipment


($357,000)


Furniture
-

$20,604


Computers, servers, tech. equipment


($29,750)



Breakeven Analysis




Operating breakeven analysis (Algebraic Estimation)


Breakeven sales level = $8,531,626


Breakeven sales volume (total units) = 54,471


Product Mix (Units)


VCR Kit


37,963


Expansion Kit


14,923


Tech. Support Service


1,583

Discounted Payback Period
-2500000
-2000000
-1500000
-1000000
-500000
0
500000
1000000
0
2
4
6
8
10
12
End of Year
NPV ($)
B.E. (NPV = 0)
NPV
Income Statement


Operating income


Costs


fixed and variable


Corporate taxes


Net earnings


EOY 1 = $2,264,200


EOY 2 = $1,796,133


EOY 3 = $1,273,679


EOY 4 = $967,216


EOY 5 = $516,715 (Project ends)


EOY 6 = ($36,864)

Gantt Chart

Engi neer( s)
Responsi bi l i t i es
14 - Jan
2 1- Jan
2 9 - Jan
6 - Feb
7- Feb
14 - Feb
15- Feb
2 2 - Feb
2 3 - Feb
6 - M ar
7- M ar
2 1- M ar
2 2 - M ar
2 0 - Apr
Anna/Suzy
Merge conf erence prog and robot prog
Anna/Suzy
Get input f ile given by client/server set up
Anna/Suzy
Creat e various point s of view depending on user
Anna/Suzy
Test ing and debuggin program f or correct ness
Anna/Suzy/Syl vest er
Finalize program/make ready f or present at ion
Saul
Research mot orola MCU and seed Model
Saul/Dyl an
Program A/D conversion on MCU
Saul/Dyl an
Program calculat ions on MCU
Saul
Est ablish SEEM model represent at ion
Saul/Dyl an
Run t est on MCU/Complet e f or f inal present at ion
Syl vest er/Anna/Suzy
Test abilit y t o read dat a f rom serial port t o program
Syl vest er
Test AF_INET socket communicat ion
Syl vest er
Dev'lop advance server program
Syl vest er/Anna/Suzy
Dev'lop prot ocol t o comm. Bt w. Client & Server
Syl vest er
Test AF_UNIX socket s f or comm. Wit h single server
Syl vest er/Suzy/Anna
Writ e client program
Syl vest er
Writ e Server program
Syl vest er
Test ing and debugging programs
Saul
Examine budget
M ork
Obt ain basic part s
Dyl an/Saul
Test single IRLED & IR Phot ot ransist er
Dyl an/Saul
Test Timer/Count er Circuit
Dyl an
Test IRLED & 3 phot ot ransist ors/w volt met er
Dyl an/Suzy/Anna
Connect t imer/Count er circuit t o mult iple LEDs
Dyl an/Sl y/Saul
Test 2 IRLEDs, Timer/Count er circuit & 3 IR Phot ot ransit ors
Dyl an/Sl y/Saul
Connect phot ot ransist or out put s t o MCU A/D input s
Dyl an/Sl y/Saul
Test single IRLED t racking
Dyl an/Sl y/Saul
Test 2 IRLED t racking
Dyl an/Sl y/Saul
Test all IRLED t racking
Dyl an/Sl y/Saul
Scale-up and product ion of prot ot ype
Group ef f ort
Final design scheme & present at ion
Conclusion


The construction of inexpensive virtual reality
hardware and rendering software is a difficult task


Our design seems well
-
suited for the task, obviating
many of the problems inherent with the technology
we employ.


There may be newfound design issues uncovered
during the physical design process. However, our
open
-
ended hardware/software design is well
-
suited to the versatile needs of such a project.

Questions?