Motion Music Controller

parisfawnAI and Robotics

Nov 17, 2013 (3 years and 6 months ago)

59 views


Ryan C.
Bergsmith

Ross Kelly

Kevin Warne


Sponsor: Steve Peralta

Motion Music Controller

Project Description


Utilize Microsoft’s XBOX
Kinect

SDK to develop a MIDI
controller with the
Kinect

sensor


XBOX
Kinect



motion/voice sensing device originally
developed for the XBOX 360


Kinect

SDK
-

provides
Kinect

capabilities to developers to build
applications with

C++,

C#, or

Visual Basic

by using

Microsoft
Visual Studio 2010



MIDI controller


(Musical Instrument Digital Interface)
Enables

electronic musical instruments,

computers

and other
electronic equipment to communicate and synchronize with
each other

Project Goal


Use the existing SDK in conjunction with our own
application to perform the essential functions of the MMC


Positional Tracking


Voice Recognition


GUI Interaction


Utilize a third party Digital Artist Workshop (DAW) to
provide recording/looping functionality


Provide the open source community with a framework for
using the XBOX
Kinect

as a MIDI controller


Develop a new and exciting method of creating music



Good Project Because…


Opportunity to learn/understand


the software development life cycle


Develop software around hardware


constraints


Makes use of newly released SDK for XBOX
Kinect


Develop a Windows application using C#/.NET


Familiarization with the industry standard MIDI protocol


Will serve as a useful reference for open source community


Unique musical creation environment

Sponsor


Steve Peralta


Provide programming
consultation



Track/report team progress



Host regular meetings




Aid in the resolution of
encountered problems


Roles and Responsibilities


Ross Kelly


GUI development and design, MIDI interfacing,
interfacing with Skeletal Recognition System



Ryan
Bergsmith



Skeletal Tracking System development and
design, MIDI interfacing



Kevin Warne


Voice Recognition System development and
MIDI interfacing


Voice Recognition System


To Dos


Develop a library of necessary


voice commands to utilize our


application


Design command
heirarchy


Test/validate correctness



Deliverable


Voice command interface which provides menu navigation,
instrument page selection, and DAW interaction


Positional Tracking System


To Dos


Develop positional indication software


Add velocity recognition and instrument


specific behaviors


Design interaction scheme with other modules


Test/validate correctness



Deliverable


Positional tracking/feedback system which generates MIDI
signals triggered by body movements



Graphical User Interface


To Dos


Design a reference instrument page


Interface with other modules


Test/validate correctness



Deliverable


GUI which displays instrument pages, provides visual feedback
of interactions, and supplies a text based menu of voice
commands and their functionality


MIDI Interaction


To Dos


Configure the
Kinect

as a MIDI device


Create a library of MIDI signals


Test/validate correctness



Deliverable


Software module which provides correct interaction
between the XBOX
Kinect

and the DAW

Summary of Deliverables


Voice command interface which provides menu navigation,
instrument page selection, and DAW interaction


Positional tracking/feedback system which generates MIDI
signals triggered by body movements


GUI which displays instrument pages, provides visual feedback of
interactions, and supplies a text based menu of voice commands
and their functionality


Software module which provides correct interaction between the
XBOX
Kinect

and the DAW


Software requirements specification, Gantt chart, non technical
user manual, final report, and a Windows application





Challenges and Concerns


Latency issues in the SDK


Industry standard for a MIDI


device is 10ms!



Third party DAW interfacing


How to implement advertised



functionality



Implementing a C# based MIDI protocol



Staying up to date with SDK releases







Milestones


Implement proof of concept


software


Generate a desired sound using the MIDI protocol


Develop a fully functional instrument page


Record/loop/playback a musical composition


Develop beta software for testing and public feedback


Present final release

Plan of Action Through 2011


Finish all proof of concept testing



Fully understand MIDI protocol/interfacing



Develop Alpha software release