Adrian Buckley
Evan
Defibaugh
John Duhamel
Alex Kaiser
Darnell Parker
Craig
Riggins
T Q
~Q
Project Objectives
-
Relay real time Nintendo 64 session data
between multiple hosts for network play
-
Provide an interface for a Microsoft
Kinect
frontend in place of tradition controller
-
Utilize non
-
obtrusive electronics
-
Minimize cost and maximize functionality
-
Power consumption is a secondary concern
System Architecture
Client 1
Client 2
Server
Nintendo 64
Server
Client 1
Client 2
Player 1
Television
Player 2
Television
Player 1
Controller Signal
Player 2
Controller Signal
Player 2
Controller Signal
Player 1
Controller Signal
N64 RCA
Signal
N64 RCA
Signal
N64 RCA
Signal
N64 RCA
Signal
N64 RCA
Signal
Client Architecture
Regular N64
Controllers
Television
ARM Cortex
MCU
N64
Controller
Signal
Kinect
USB Signal
Digital
RCA Signal
Digital
Controller Signal
Ethernet IO
Microsoft
Kinect
Analog RCA
Signal
MCU takes input from 4 N64 controllers
Microsoft
Kinect
can track 2 people
Server Architecture
ARM Cortex
MCU
Digital
Controller Signal
Digital
RCA Signal
Ethernet IO
Nintendo 64
N64 Controller
Signal
Analog
RCA Signal
Can communicate with 1
-
4 clients
Moves controller signal from client to N64
and RCA signal from N64 to client
Unified Hardware
N64 Controllers /
Microsoft
Kinect
Television
ARM Cortex
MCU
Controller Signal
RCA
Signal
Controller
Signal
Ethernet IO
RCA Signal
Nintendo 64
Controller
Signal
R
CA
Signal
1 hardware architecture will be able to
operate as either the client or the server
Advanced System Architecture
Client 2
Client 1 /
Server
Nintendo 64
Client 1 /
Server
Client 2
Player 1
Television
Player 2
Television
Player 2
Controller Signal
Player 2
Controller Signal
Player 1
Controller Signal
N64 RCA
Signal
N64 RCA
Signal
N64 RCA
Signal
N64 RCA
Signal
Kinect Integrated Peripheral
Kinect Processor Linux Microcontroller
XBOX Kinect
Processor
Linux
Microcontroller
Kinect Sensor
I/O
2 Cameras (RGB and IR Depth)
IR Output
4 Array Microphone
USB Port
Possible Implementations
ROS:
BeagleBone
VS C# 2010: Netduino
VS C#
2010: PC
Implementing Kinect
via ROS on Beaglebone
ROS (Robot Operating System) is an open
source project focusing on interfacing
microprocessors with robots (in our case the
Kinect and a peripheral Linux microcontroller)
Kinect Processor
Control Node
The Netduino operates on Microsoft Visual
Studios C# 2010 via Net. Micro workforce
Implementing Kinect
via VS C# 2010 on
N
etduino
Minor Hiccups in Progression
Using the Robot OS
ROS tutorial has
incorrect/missing
info
Open source, thus
compatibility issues
from node to node
Unintuitive to
navigate
Using the Netduino
•
Netduino requires
Net. Micro
framework, where
Kinect requires Net.
Framework
•
Trouble converting
from standard to
micro
There is no problems implementing Kinect via PC
Last Resort and Additional info
As a last resort the Kinect can be routed
through a Windows PC for processing then
sent to microcontroller.
More ROS info
http://www.ros.org/wiki/kinect
(Install ROS)
http://www.ros.org/wiki/ROS/Tutorials
(ROS Tutorial)
http://beagleboard.org/bone
(
Beaglebone
Description)
http://www.ros.org/browse/list.php
(ROS software)
More Net. Micro workforce
http://docs.com/BTN4
(Installing
OpenNI
)
http://www.netduino.com/netduino/schematic.pdf
(
Netduino
Schematics)
http://informatix.miloush.net/Microframework/Arti
cles/WpfEmulators.aspx
(
.net
micro install)
http://informatix.miloush.net/microframework/Arti
cles/Reflector.aspx
(
.net
micro reflector)
http://netduino.com/netduino/specs.htm
(
Netduino
Specs)
Gesture and Speech Algorithms
Joint Data is returned in X, Y, Z, and W
(confidence) data
Evaluate position of joints relative to another
and trigger button pressed event
For audio grammar files must be written
Compare text output of audio and set trigger
for the controller or the sensor as being used
Output control trigger and button triggers
Kinect Processor Interface
The Processor will output triggers in the form
of buttons and sensor/controller triggers for
both players one and two
The triggers will be relayed to the Linux
microcontroller that will convert button trigger
to N64 controller output format and use the
control trigger to decide if forwarding the N64
controller data or sensor converted data
N64 Controller Tutorial
How the N64 Controller Sends Its Data
The N64 controller only has three pins: Ground, data, and
VCC. It takes a +3.6V supply.
The N64 uses a protocol in which bits are sent serially to and
from the controller as 4µs
-
wide pulses. ‘Low’ is 3 µs low and
1 µs high, and ‘High' is 1 µs low and 3 µs high.
N64 Data Polling
The N64 continually polls the controller using an 8
-
bit
command (0x01), and the controller reports the state of its
buttons and analog joystick with a 32
-
bit response: 16 bits for
the buttons, signed 8 bits for horizontal joystick position, and
signed 8 for vertical. Each bit packet ends with a stop bit (1).
Button Configuration
The buttons are mapped to their respective bits as follows:
Sample Joystick Positions and respective bits:
Bit
0
1
2
3
4
5
6
7
Button
A
B
Z
Start
D
-
Up
D
-
Down
D
-
Left
D
-
Right
Bit
8
9
10
11
12
13
14
15
Button
Null
Null
L
R
C
-
Up
C
-
Down
C
-
Left
C
-
Right
Left Extreme
1
0
0
0
0
0
0
1
Left Off Center
1
1
1
1
1
1
1
1
Right
Off Center
0
0
0
0
0
0
0
1
Right Extreme
0
1
1
1
1
1
1
1
Up
Off Center
0
0
0
0
0
0
0
1
Down Off
Center
1
1
1
1
1
1
1
1
Joystick Off
-
Center Left & Down
The Translator Box
The
Kinect
will take player movement/sound data and
send over button and trigger data to the “translator box,”
which then converts that data into the 64
-
bit
struct
format shown below.
There will be a physical master switch on the translator
box, to switch between
Kinect
control and N64 controller
control. When this switch is set to “
Kinect
,” the
translator will await trigger and button data from the
Kinect
to process.
If the switch is set to “N64,” the translator will need to
keep track of one or two controllers, simultaneously
converting their button presses into the shown format,
but with the “More Buttons” bit and time delay
consistently set to zero.
Button Data
Joystick
Data
Which Player?
More Buttons?
Time Delay
16
bits
16 bits
1 bit (0 or 1)
1 bit
Int
(30
-
bit)
Server to Translator to N64
Another Translator Box is needed to translate the
Server’s data into N64 controller data for the N64.
This translator will read in the controller data, and send
the correct button sequence via serial ports to the N64,
correctly timed. If the N64 polls the “controller,” but the
translator has not received button data from the server,
the translator will automatically respond with a 32
-
bit ‘0’
analog stream and continue doing so until new button
data appears.
Audio/Visual
Signal
The video output of the Nintendo 64 carries
visual information
via an analog signal, plus
two synchronizing signals for aligning the
video signal on a television.
The audio
output, also analog,
is split into
two signals for stereo speakers in a
television.
Audio/Visual
Waveforms
Video
Audio
Audio/Visual
Sampling
The audio and visual output of the Nintendo
64 must be digitized and formatted
for
input
into the server for transport over the internet.
Once the
audio/visual
information is sent
over
the
internet it must be converted back
into an analog signal to be displayed on a
player's television
.
A Series of Tubes
–
Hardware
Initial design and testing will be done on an
A
rduino
with an
E
thernet shield.
Ardiuno
is easy to program to send and receive
data packets.
After getting a fully functional system with the
Premade hardware we will move on to a
custom PCB with a Linux OS.
Our Internet Objectives
We need the data received from the server
to be real time
If there is missing data we just move to the next
data packet.
We know that the receiver is ready to receive
packets.
TCP/UDP
TCP/IP uses an established connection
between sender and receiver.
This requires an acknowledge signal from the
receiver which takes time.
UDP instead sends packets in a continuous
stream and doesn’t care if packets are
missing on the receiver side.
Instead of waiting for acknowledgement UDP just
tags the packet with the correct receiver IP
address and sends it on its way.
UDP in depth
UDP sends data with a header, and within
the header a 2byte portion telling the
receiver how long the data section will be.
This 2byte section allows for a maximum of 64Kb
section of data which is more than enough for
sending the n64 signal and frames of an RCA
signal.
The other parts of the header are receiver IP and
port number.
Putting it all Together
Using UDP because we need a real time
connection and don’t mind dropped packets.
Using a static IP for our server so that we
have something to aim at for our UDP
packets.
Using
Arduino
and Ethernet shield to
prototype and onto
custome
PCB once all
kinks are worked out.
Project Schedule
Longest Design
Period
11 Days for Testing
Technical Reference Manual
37 Days for Integration
Division of Labor
Task
Adrian
Evan
John
Alex
Darnell
Craig
Program Mgmt &
Documentation
P
S
S
Kinect
Programming
S
P
S
N64 Translation
S
P
S
Ethernet Interface
P
P
Cont牯lle爠偲g牡浭mng
S
S
Audio/Video
卩gnal
Conve牳ron
S
S
丶㐠
Ga浥play
A捴ion
P = Primary
S = Secondary
Advanced Parts List
TOTAL ESTIMATED PROJECT COST = $1,074
*NOTE*
–
ALL EXPENSES PAID TRIP TO LAS VEGAS FOR “ELECTRONICS
SEMINAR” NOT INCLUDED
Goals
Primary
:
•
Control Nintendo 64 game exclusively via an XBOX
Kinect
Secondary
:
•
Have one N64 controller communicate with one N64
console over the internet
•
Control the N64 game using either the Kinect or the N64
controller remotely
Tertiary
:
•
Play a complete N64 game with 2 players at one location
using the
Kinect
, and the other 2 players at a different
location using N64 controllers
•
Allow functionality with a variety of different controllers
Milestone 1 Objectives
Kinect Translation
•
Kinect translator sends simple gesture data to
the Linux µController, in correct format
Nintendo 64
•
Send properly timed Nintendo 64 analog signal
from the µController to the N64 console
Network Configuration
•
Send data packets across the network to and
from each user using the µController
Audio/Video
•
Convert analog input into a digital code
Milestone 2 Objectives
Kinect
Determine ideal
Kinect
gestures for
satisfactory user interface
Transmit verbal commands through
Kinect
Network Configuration
Send and receive N64 and audio/video
signals across server
Audio/Video
Output signal from server to television
Risks and Contingency Plan
Risks
Syncing the N64 timing sequence with the
Netduino
microcontroller’s internal timer
Busy internet path/lost data packets
Delay between
Kinect
/ N64 input and television
output
Data corruption or potential injury resulting from
falling or player collisions while playing with a
motion sensitive controller
Solutions
Use only N64 controllers (no
Kinect
) for game
control
Leave out server / direct connection from
microprocessor to N64
User plays video game at own risk!!
Questions???
Enter the password to open this PDF file:
File name:
-
File size:
-
Title:
-
Author:
-
Subject:
-
Keywords:
-
Creation Date:
-
Modification Date:
-
Creator:
-
PDF Producer:
-
PDF Version:
-
Page Count:
-
Preparing document for printing…
0%
Σχόλια 0
Συνδεθείτε για να κοινοποιήσετε σχόλιο