ELECTRONIC CONDUCTING SYSTEM

photohomoeopathAI and Robotics

Nov 24, 2013 (3 years and 8 months ago)

83 views

ELECTRONIC CONDUCTING
SYSTEM

Kenzo Abrahams


Supervisor:
Mehrdad Ghaziasgar

Co
-
supervisor: James Connan


Assisted by: Diego Mushfieldt

Overview


Introduction


High Level Design


Implementation


Testing and Results


Demo


References


Questions and Answers

Introduction


Interactive Conducting System


Tracks hands using webcam


Real
-
time alterations depending on hand gestures


Change volume


Change tempo


High Level Design


The solution can be broken up into 3 parts


Input


Image processing


Adjust music



Implementation

Extract

F
rames

Detect face

HSV

Region of
interest

Extract Hue
component

Implementation

Draw histogram

Track right hand
(control tempo)

Track both hands
(change volume)

Testing and Results


Three tests were performed


Response Time Test


Performance Test


Robustness Test



Response Time Test


Each gesture is played for 20 seconds


iteration

Number of noticeable delays

Gesture 24

Gesture 34

Gesture 44

1

2

2

2

2

2

1

3

3

1

1

2

Average

2

1

2

Performance Testing


Each section of the algorithm was timed

0
5
10
15
20
25
Extracting Frames
Face detection
Convert to HSV
CAMShift
Total
Time (ms)

Sections of Algorithm

Time taken for each section of the algorithm

Time (ms)
Robustness Test


15 users of different ethnicity


Check number of times the system loses tracking

User number

Number of losses

Black

Coloured

White

Total

1

0

0

1

2

1

1

1

3

0

0

0

4

0

0

0

5

1

0

0

Total

2

1

2

5

Demo


Demo Overview


Track right hand


Change the tempo of music


Track left hand


Decrease volume of symphony


Increase volume of one ensemble


Show that all the gestures work

References


O'Niel, D L. (2008).
Music Theater Jobs.

Available: http://www.ehow.com/list_6547349_music
-
theater
-
jobs.html. Last accessed 29th March 2012.


Bradski, G Kaehler, A. (2008). Getting to Know OpenCV. In: Loukides, M
Learning OpenCV
. United
States of America: O’Reilly. p31
-
87.


Gibson, J. (2009).
RTcmix Tutorial.

Available: http://iub.edu/~emusic/509/rtcmixtut.htm. Last
accessed 10th Sep 2012.


Nakra, T., Ivanov, Y., Smaragdis, P., Ault, C.: The USB Virtual Maestro: an Interactive Conducting
System, p.250
-
255, NIME2009 (2009).


BORCHERS, J., HADJAKOS, A., AND M
¨
UHLH
¨
AUSER, M. MICON (2006) A music stand for interactive
conducting. Proceedings of the 2006 conference on New interfaces for musical expression, p254

259.


Viola, P. Jones, M. (2001). Rapid Object Detection using a Boosted Cascade of Simple Features.
Conference on computer vision and pattern matching 2001
.


Lansky, A . (2005).
An Open
-
Source, Digital Signal Processing and Sound Synthesis Language.

Available: http://rtcmix.org/. Last accessed 27th May 2012.


Nokia. (2009).
Qt.

Available: http://qt.nokia.com/products. Last accessed 10th Sep 2012.


Questions and Answers