Follow the Keys:

mattednearAI and Robotics

Dec 1, 2013 (3 years and 6 months ago)

85 views







Follow the Keys:

The Virtual Instructor


Leah Jones
(
997554533
)

Kyle St Jacques

(
997418012
)


Final Report


CCT375H5


Dr. Chattoe


March 23
, 2012



Jones and St. Jacques


2

Executive Summary

Learning sight
-
reading is frustrating and can take years for the casual piano
player.
This daunting task may deter prospective players, robbing them of the mentally
stimulating and emotionally therapeutic benefits playing piano
.
The target users include
those aged 30
-
45, with limited knowledge of music and music theory (or those who

used
to hold these abilities, but lost them years ago), and who wish to learn how to play songs
on the piano, rather than master the instrument.


Our augmented
-
reality solution overlays a virtual keyboard onto a physical piano.
The virtual image will disp
lay the keys pressed, and will allow the user to copy and play
along with the sequence. The virtual keyboard will be controlled by the user and will have
the ability to adjust tempo and loop specific sections. What makes our concept unique is
the focus on
learning individual songs rather than learning theory. This is a more relaxed
and casual approach, more appealing to our users. Our concept also allows users to build
on the songbook themselves (open source), allowing for more current pieces, and a
greater

number of teaching possibilities


We conducted a second evaluation focusing on the system’s skill level test
component. Though a series of paper prototype and questionnaires, the test concluded
that users prefer having more information displayed with
greater option to calibrate the
appropriate difficulty. In other words, the participants would understandably prefer to
have more control over their rate of learning than the computer.


Future plans include further developing the song playing and
downloading system,
and the auditory feedback and instructional elements.

Jones and St. Jacques


3

Index:

Section 1: Application, Problem and Target User Community

………
…………
….
…..4


The Problem


The Users


The Solution


Interaction Scenarios


Section 2: Design Concept
………………
.……………………
…………………………….……………….7


Existing Implementations


Our Design


Section 3
: Prototype and Evaluation Results
……..……………………………………………..14


Second Stage Prototype


Evaluation Results


Section 4:
Recommended Design and What to Do Next
…………………
…………………18



Appendices
…………
………………………………………………
……………………………………………….20


A:
Survey Results


B: Design Details


C:
Second Prototype


D:
Second Evaluation


E: Questionnaires


F: Questionnaire Results



Bibliography
………………………
………………………………………………………………………..………35
Jones and St. Jacques


4

Section 1: Application Problem
and Target User Community

The Problem

Research has shown playing the piano has many personal benefits for adults,
including an increase in “self
-
confidence, self
-
discipline, self
-
esteem and self expression”
(Jutras 107
, 2006
), and can even lead to “self
-
ac
tualization” (Jutras 106
, 2006
).

However, le
arning sight
-
reading is frustrating and can take years for the casual piano
player. For a beginner looking to play casually, sight
-
reading may not even be necessary
as a “certain amount of technical mastery of a
n instrument must be attained in order for
instrumentalists to sight
-
read” (Wristen 45, 2005).

Even for a practiced player, “t
he need
to look at the musical score must be balanced with the need to look at one’s

hands”,
allowing for the


potential danger of

losing one’s place in the score a
s one’s eye moves
and refocuses” (Wristen 46, 2005).


In a study of personal opinions regarding the benefits of sight
-
reading, one
musician claimed
,


r
eading music is a fantastic and useful skill and well worth learning,
but being able to read it correctly the first time is less important than being able to play it
with feeling on the umpteenth occasion” (Bryce and Fortner 15, 2007).

As team members

w
e both play piano, and though we have an appreciation for the
benefits of

sight
-
reading, we agre
e it can be very difficult.
Learning a song by ear or by a
tutorial is much more beneficial for the un
-
practic
ed player, especially with
popular/modern songs
.
The incredible amounts of video tutorials on Youtube are further
proof of
the use of visual aids in learning piano, and the number of people out there who
are unable to read sheet music and rely on this visual type of learning.


Jones and St. Jacques


5

We have also discussed
the problem with

subject matter expert

Marlene Borsella
.
Ms. Borsella is

56 ye
ar
s

old
, and has

been teaching piano to students for 30 years
.

She
agrees sight
-
reading can be difficult to learn at a later age, and a visual method of
learning is “
a faster source for a lot of people


and can be

“a great thing

because less is
more


the
basics in a box
” (Borsella, 2012).
This daunting task may deter prospective
players, robbing them of the mentally stimulating and emotionally therapeutic benefits
playing piano.


Users

When speaking with our expert, Mrs. Borsella suggested,


the number one

type of
person that’s going to go after that are adults who have always in their whole life wanted
to play”

(2012).
Therefore, the users we are focusing on are those in the 30 to 44 year
old age bracket, with limited playing ability.

These are individuals

who either want to be
able to play casually without going through the hassle of learning piano theory, which can
be both difficult and time consuming for those learning at a later age.

A majority

of those
interested will most likely be those who have prev
iously lost the ability to play, as studies
found that fifty
-
six percent of those “who stopped playing their instrument between the
ages of 12 and 14 stated that they had forgotten so much that they no longer played”
(Cooper 157
, 2001
).
An examination of l
iterature reveals this adult age group often play
for personal enjoyment, and noted “personal pleasure was an important motivating factor
for adult piano students” (
Jutras 98
, 2006
)
.
This is also an age group that prefers “self
Jones and St. Jacques


6

directed learning” and “can

manage delayed gratification” (Billings 104), which changes
the types of motivators necessary for the system to work.


The Solution

Our system teaches only through guided playing and eliminates the necessity for
sight
-
reading.

Though sight
-
reading is esse
ntial to skilled pianists (Wristen 45, 2005), it
can be likened to learning a new language and is far too time consuming and difficult for
our user group. Our system focuses on learning piano on a song by song basis, with
extended user control to allow use
rs to learn at their own pace. Our system will also
customize to the user at the start of their first session through a series of tests intended
to determine their musical skill level; and adjust tempo and song selection accordingly.
This all makes for a u
ser
-
centred experience, uniquely catered to those casual players
who wish to learn individual songs rather than seeking to ‘master’ the instrument.



Interaction Scenario

The following is a simple fictional scenario to demonstrate the c
oncept (the full
ste
p
-
by
-
step scenario can be found in appendix B)
and its target audience.

X sits down at the piano bench and puts on all the necessary equipment. X has
participated before. Using a mouse on the piano bench, X selects her user profile from a
list that appears

across her glasses. X selects one of the songs

from the personal
songbook that appears
, and using the indicated piano key, plays a preview of the first
sequence in the song. This preview both visually displays the notes on the keys and plays
Jones and St. Jacques


7

a sound recor
ding of the sequence in time. X then chooses to begin practicing, and is
visually counted down before the keys begin to light up in the same sequence. X plays
along, and when she finishes the sequence, she is given a score of 88%. X moves on to
the next se
quence, but after playing it, only receives a score of 54%. X decides to slow
down the tempo of the song, and try again. Once satisfied with her learning, the system
asks X to repeat everything she has learned so far (ie: both sequences together) before
mo
ving on to the next sequence. This is repeated until the song’s completion. When X
plays all sequences well (relative to her chosen difficulty settings) she is ‘rewarded’ and
the system allows X t
o add a new song to her songbook
. She clicks the mouse to br
ing up
the menu, and selects a new song.


Figure 1: the user’s view using the transparent AR glasses


Section 2: Design Concept

Existing Implementations

Other augmented reality implementations we have discovered are relatively simple,
and are designed as
more ‘one time experiences’ than the full narrative learning
experience we intend to create.

Jones and St. Jacques


8

Once concept is called
Handel
, an augmented
-
reality system to help with song
memorization that relies “on hand movements to trigger an augmented
-
reality overlay
on
to the user’s hands during piano practice” (Cheng and Robinson 17).

This system is not
designed to teach new players, but rather to help seasoned players with their
memorization of piano pieces. It uses a “wearable computer” to allow the user to look at
on
e of their hands as a cue for the system to overlay bits of sheet music when the

player’s memory fails (figure 2
), using hand tracking systems to determine notes played
(Cheng and Robinson 17).


Figure 2
: Handel’s display


The concept we found most relev
ant to our idea is a
“Markerless Augmented
Reality Based
Piano Teaching System” (figure 3
),
and allows users to follow superimposed
‘virtual fingers’. The paper discussing this project focuses more on the technical aspects of
the system, specifically the w
ay the “geometrical feature of the keyboard is made full use
of to replace of the marker in traditional AR system” (Zhou et. all 47, 2011), a technique
that seems to us, much more appealing than using markers.

We decided to adopt this
method of topographic
al tracking from, where a “binarization of the original image is
made based on the fact that there are only two colo
u
rs associated with piano keyboard,
black and white” (figure 4) (Zhou et. all 49, 2011). This technique saw positive results in
Jones and St. Jacques


9

its ability
to track the head movements of the user (Zhou et. all 50, 2011), and adjust the
AR projections accordingly
.


Figure 3
: Markerless tutor

Other systems much more familiar come in the form of an iPhone application.
Applications such as “PianoChords”, “Piano
Pinc
h” and “Amazing Piano” (figure 4
)

use the
touch technology to teach players basic notes and simple tunes by either having the
proper keys light up or change colour, allowing the user to follow along (
20 Best
Applications for iPhone,

2010). Though entertaining, we determined these apps are to
“follow along” to offer a learning experience, and would not transfer as well to a real
piano if the user desired. The applications also aim to teach notes and chords


elements
of theory that ar
e unnecessary for the casual player. Furthermore, the medium itself is
limiting, as the iPhone’s screen is a small standard size, and so could only offer an few
octaves of playing space at the most.


Figure
4
:

‘PianoChords’, ‘Piano Pinch’, and ‘Amazing

Piano’ respectively

Jones and St. Jacques


10


Our Design


The physical design

of the system (as seen figure 5

below) is quite straightforward.
The user sits at a digital piano, wearing transparent glasses and high
-
quality headphones.
A wireless mouse for menu navigation is placed

on the bench beside the seated user

(see
appendix B for more specific design information)
.


Figure 5: physical setup


As the user is seated for the entire experience, common issues like encumbrance
and throughput do not affect the user. Most of our
design decisions were based on things
like visual and aural quality, as the system is designed to allow the user to remain in
session as long as they would like, meaning we need higher quality display to avoid things
like visual exhaustion or eye strain.

Jones and St. Jacques


11

O
ur system architecture is also quite straightforward.
Learning a song on piano is
basically achieved through repetition.
This means the system must include enough detail
so the user can understand the first time without external help, but not too detailed
so as
to annoy the user after they become comfortable with it. In other words, the

system must
be able to be easy to navigate through over and over again, and must be simple enough
for the user to move through smoothly

without getting annoyed by instructio
ns that are
too long, even after going through it the twentieth time.
The basic architecture of th
e
system can be seen in figure 6

below.



Jones and St. Jacques


12

Figur
e 6
: System Architecture/Use Case
.

Red: computer determined. Blu
e: user choice. Black: next step (no option)



This simple structure keeps the learning pace user
-
determined, allowing for a more
self
-
directed approach to learning piano that better fits the “self directed learning”
approach (Billings 104) preferred by our target age group. This repetition also allow
s for
two kinds of reward systems: system
-
based rewards (ie: unlocking new songs), and self
-
gratification reward, where


as in the case of learning any instrument


learning to play a
song, and the pride that comes wi
th that, is its own reward. This is pa
rticularly true for
our age group, where studies have found that adults “who continue to play have done so
because of personal pleasure and satisfaction” (Cooper 167
, 2001
).


The user is given full control of the song
-
playing mode, accessible through a mai
n
menu brought up by clicking the mouse (figure 7). This menu allows the user to further
customize their learning experience, letting them learn at their own pace and further
reducing the frustration that can come with not feeling in control (frustration t
hat could
lead to the user abandoning the program).

Jones and St. Jacques


13


Figure 7: Game
-
play Menu


One element of our design that is quite unique is in regards to the content. Rather
than focusing on teaching theory elements of piano, our system teaches on a song
-
by
song
basis. The main content we must consider is the music itself. In order for the songs
to be learned by the users, they must be first either played out on a piano that has the
ability to record the notes that are being played, or translated from sheet music
by a
computer.

Having the capability to scan an original piece of sheet music and then transferring
it to our AR device has the potential to be an impressive learning feature. Software
presently exists such as SmartScore® by Musitek® (Finale, 2012) that al
lows a user to
scan an original document and transfer it to a digital staff writing/editing program.
Jones and St. Jacques


14

Granted, this method is much more prone to errors than other aspects of the system, but
it would still allow the user to have greater choice


even knowing

the potential for errors.
One plan we have for possible future expansion is to create and open source website for
our experience, where players can upload and download user
-
created content which is
legible to the system. This would provide the user with a

much larger song library. This
larger database of songs would provide better accommodation of taste and ability for
system users, and by allowing the users to choose the songs they are working towards,
enhances the experience’s reward system. However, thi
s method of collection is both time
consuming and user
-
dependant, and so we would program the system with a set selection
of songs, all pieces without copyright issues. This will not be too big of an issue, as many
popular piano pieces are in fact quite ol
d, and the composers who wrote them are long
since passed away (i.e. Mozart, Braque). This follows the copyright laws of Canada, where
“t
he term for which copyright shall subsist shall

be the life of the author, the remainder
of the calendar year in which

the author dies, and a period of fifty years following the end
of that calendar year
” (
Copyright Act
,

2005). We would also attempt to gain the rights to
use more current music through deals with those who hold the rights to the songs.


Section 3: Prototyp
e and Evaluation Results

Second Stage Prototype


For our second evaluation, we decided to prototype the ‘Level Assessment Test’
subset of our design.
Our motivation for choosing this subset were two
-
fold: we chose the
section in reaction to the results of
our previous report, and this subset will be the first
Jones and St. Jacques


15

interaction our user has with the system, and so feedback is essential in order to engage
the user the first time they use it.

In our previous report, we presented the subjects with a prototype of the
‘Play
Song’ subset, and found few problems with its organization. We believe this is due to the
simple nature of this section. It is relatively easy to understand the concept of following
along with notes, and making a choice of what to do next. Therefore,

we have
encountered no drastic changes to this subset, and believe the section that was in more
need of evaluation is the ‘Level Assessment Test’.

We also chose this subset because, as the user’s first interaction with the system, it
will prove to be the
most challenging for the user to understand. This section of the
system both judges the users playing ability (assigning them a level from 1
-
10) and
provides the user with a practice tut
orial.

We developed a ‘digital paper’ prototype as a series of images
and animations
illustrating what the user would see at a given time. We present the illustrations in the
order they would be presented in the actual scenario, allowing the user to make choices
based only on what they see

(see appendix C for more examples)
.

Using this ‘wizard of oz’
technique forces the subjects to navigate the system themselves, guided only by the
simple instructions that appear in the images. This mimics the way they would navigate
the system when completed, allowing the feedback to be as
relevant to the final system
as possible.

When the tests were accompanied by two questionnaires, one pre
-
test and
one post
-
test (
which
can be seen in appendix E).

These questionnaires used a 10
-
point
Jones and St. Jacques


16

scale to assess musical ability, and a tradition
al 4
-
poi
nt LIKERT
scale to answer questions
regarding elements of the test sequence (figure 8).


Figure 8: LIKERT scale question example


Evaluation Results

The subjects we recruited to participate in the second stage prototype evaluation
(named Subjects 5
-
8) are

all within or very close to the stated age demographic of our
project. Once again the participants were equally represented by both males and females
and each participant had been found to have a low to middle range for both musical
proficiency and knowle
dge. The participants were chosen from our hometowns of Burford
and Port Elgin, Ontario. Each participant was quizzed before being presented the second
-
stage prototype in order to evaluate their current musical knowledge and abilities (see
Appendix F).


T
he participants then navigated their way through the prototype images and were
asked to fill out the ‘post
-
use’ questionnaire. The questionnaire addressed topics including
their perceived comfort level taking private piano lessons, their understanding of o
ur
testing system, their preferences concerning the test options, and to what degree this
device would encourage them to play piano. The answers were based on a four point
Likert scale


disallowing for a neutral response


to evaluate the degree to which
they
agreed with content specific statements.

Jones and St. Jacques


17

We found that among this test group, the participants universally identified as
being comfortable taking private piano lessons. Besides this question however, the
opinions between participants varied. When aske
d to identify how intuitive they
considered test sequence (figure 10), all except subject 5 gave a positive value; subject 5
identified as having some reservations only because of the nature of the prototype; he
stated that a real life test sequence would
probably yield a positive result.


Figure

10

The general finding concerning the format of the test (i.e. the information provided
during the test sequence and how it would conclude) found that users prefer having more
information displayed with greater
option to calibrate the appropriate difficulty (figures 11,
12). In other words, the participants would understandably prefer to have more control
over their rate of learning than the computer.

0
1
2
3
4
Subject 5
Subject 6
Subject 7
Subject 8
The test sequence was intuitive
and easy to understand


Jones and St. Jacques


18


Figure 11


Figure 12


Section 4: Recommended Design and Wha
t We Would Do Next

Future concepts to be prototyped and tested include the song saving/download
system and auditory instructions/feedback system. One option that we have decided that
should be included is a ‘Quick
-
Play’ option; which effectively
circumvents the progressive
instructional component to the system and allows the user to play any song they wish.
0
1
2
3
4
Subject 5
Subject 6
Subject 7
Subject 8
I would prefer to see my
percentage displayed rather than
have the program simply stop the
test


0
1
2
3
4
Subject 5
Subject 6
Subject 7
Subject 8
I would want to skip these tests
and simply go at my own level

Jones and St. Jacques


19

This is of course the main purpose of our system (to learn and play individual songs at the
users’ discretion), the main change is to allow th
e user to access any song


even if the
song is available to unlock within the instructional component. For example, if a user
wishes to play Beethoven’s ‘Fur Elise,’ (which may be available to unlock at one of the
higher difficulty levels), they could cir
cumvent the process and simply access it through
the quick play option.

One subject suggested including audio feedback, including a digital tutor that can
give tips and suggestions to users regarding things like timing, finger position and
etcetera. We pla
n to add this to our system, but intend to make it optional (ie: a menu
option). We made this decision so as to give the user further control over their learning
experience, so if the user considers the tutor to be too discouraging or repetitive they can
e
liminate the feature to
relieve

some of the perceived pressure.


Jones and St. Jacques


20

Appendices

A: Survey Results

Handel

“While the pianist plays the piece, Handel doesn’t overlay anything on the pianist’s
heads
-
up display until it sees skin colour. When Handel detects skin,

it assumes that the
pianist is looking down at the hands” (Cheng and Robinson 17).


(Cheng and Robinson 17).

Markerless System

“The system finds out the pre
-
defined keyboard from the captured image by
picking out all the possible contours of the keyboard
and matching the number of black
keys. After tracking the keyboard, we establish the three
-
dimensional

coordinate system
based on the keyboard, and then calculate the position of each piano key. According to
Jones and St. Jacques


21

the piano score imported, virtual fingers can pl
ay on the real keyboard.” (Zhou et. all 47,
2011).


(Zhou et. all 51, 2011)

Barriers/Issues to Consider



We must consider the stigma that is often attached to practicing an instrument,
and f
ind ways to make it enjoyable. An examination of the l
iterature
claims the
“enjoyment of lessons, enjoyment of practicing, and continued playing were significantly
related to perception of skill particularly for those who took lessons as children” (Cooper
162, 2001). Cooper found that “subjects who rated their keyboard

skills as "very good"
and "pretty good" during childhood years were more likely to report enjoying lessons,
liking to play better, and enjoying practicing” (Cooper 162, 2001). In order to circumvent
this problem, we must find ways to make the experience a
s enjoyable as possible, and
prevent the user from becoming frustrated at their perceived lack of skill.


Jones and St. Jacques


22

B: Design Details

Interaction Scenario

1.

User “X” sits down at the piano bench and puts on all the necessary equipment. X
has participated before. Using

a mouse on the piano bench, X selects her user
profile from a list that appears across her glasses. A songbook appears on the
glasses.

2.

Using the mouse, X selects one of the songs with the curser that appeared on the
screen and clicks it. The menu
disappears, replaced by 1/8 (indicating the user is
on sequence 1 of 8 total).

3.

A blue key is superimposed on one of the centre piano keys (this occurrence will
be hereafter referred to as they key “lighting up”). Text ‘above’ the blue key
appears reading ‘
Preview’.


4.

X presses the ‘Play Preview’ key and a short musical sequence (ie: the first section
of the piece) is visually played out on the keys as they “light up”, supplemented
by the sound of the music itself. When the sequence is complete, the ‘Preview

key lights up, and a second key to the right lights up in green: ‘Begin’.

Jones and St. Jacques


23


5.

X selects ‘Begin’ and a selection of keys indicating her hand’s starting position
light up. Numbers count down on the screen from 3, accompanied by the clicking
sound of the metr
onome to count X in. The keys light up in the same sequence as
in the preview, without the automatic sound. X follows along, playing the keys as
they light up, and the piano outputs the sound of the notes X plays as she plays
them through her headphones.

6.

W
hen the sequence is finished ‘88% Correct’ appears above the keyboard,
indicating how well X did in that sequence. Since X did relatively well, three
options now appear on the keys: ‘Play Preview’, ‘Begin’ and ‘Next Sequence’.


7.

X selects ‘Next Sequence’,
and ‘Preview; is displayed alone. The sequence
indicator on the top left now changes to ‘2/8’.

8.

X repeats steps 4
-
5 for sequence 2

Jones and St. Jacques


24

9.

‘54%’ appears on the screen. X thinks the sequence moved too fast for her level,
and so uses the newly indicated keys to reduc
e the tempo of the song. The slider
on the screen reacts, moving to the left and the sequence moves at a slower
tempo.

10.

X repeats steps 4
-
5

11.

Four story options appear on the screen: ‘Preview’, ‘Begin’, ‘Next Sequence’ and
‘Play from Start’. X selects the lat
ter, is given the countdown, and the sequence
plays from the beginning with X following along, playing through all sequences
she has played so far. As she moves from sequence to sequence, the sequence
indicator changes to reflect which sequence she is curr
ently playing.


12.

These steps are repeated until X successfully plays all sequences. Text on the
screen appears saying ‘New Song Achievement’, and a song menu returns,
allowing X to select a new song to add to her playlist.

C: Second Prototype

Input

Jones and St. Jacques


25


The th
ree methods of input we chose for our system are a digital piano keyboard
with weighted keys and a standard wireless computer mouse for user interaction, and an
optical tracker.

We chose to use an electric piano rather than a traditional piano as it allows

the
system to track the user’s key inputs in a similar way to a computer keyboard. This is a
much simpler and more effective method of assessing the user’s ability than using
microphones to asses the sound of the notes produced (which can be subject to
in
terference), or some sort of trackers on the user’s hands (which, for a piano player, can
be very encumbering). This method of input also allows the system to use the keys for
other aspects of the program (ie: using a key as a ‘start’ button).


The wireles
s mouse is used to navigate more difficult menus such as song selection
and system preferences. We chose the computer mouse for its familiarity, and
convenience as it can be kept and used on a piano bench with no encumbrance issues.


The videometric (optical) tracker is head
-
based, and ‘sees what the user sees’,
allowing the system to track the user’s head movements and adjust the augmented
display accordingly. As this kind of tracking usually has correspondence issues and can be
slow,

it is best to use “distinct landmarks” to reduce the amount of computation necessary
(Sherman and Craig 83, 2003). Luckily, the keyboard of the piano itself


with its keys
uniform in layout and size


already acts as a series of landmarks. We take this m
ethod of
topographical tracking from the ‘Markerless’ system mentioned above, where a
“binarization of the original image is made based on the fact that there are only two
colours

associated with piano keyb
oard, black and white” (see below
) (Zhou et. all
49,
Jones and St. Jacques


26

2011). This technique saw positive results in its ability to track the head movements of the
user (Zhou et. all 50, 2011). The nature of our system eliminates many problems typically
associated with optical tracking; limited workspace is not an issue a
s the user is seated,
and it is this limited workspace that rids the system of correspondence and shadowing
problems.


(Zhou et. all 49, 2011).

Output


We chose to use a see
-
through head display and headphones for the visual and
aural displays of our syst
em.


See
-
through head
-
based displays are one of the most common displays used in
augmented reality, “using lenses, mirrors and half
-
silvered mirrors to overlay an image
from a computer onto a view of the real world” (Sherman and Craig 155, 2003). These
giv
e a 100% field of reference, and the mental immersion relies mostly on head
Jones and St. Jacques


27

orientation. The only real issue with this display in regards to our system is lag, and eye
fatigue. However, these can be avoided with faster processors, and higher
-
resolution
scr
eens. These are more costly, but necessary for the system to function, and with only
one user using the system at a time (therefore only needing one headset per system),
cost is less of an issue. Once again, other problems commonly found in head
-
based
disp
lays are eliminated with our system; encumbrance from wires and the limited field of
view are non
-
issues as the user is sitting down and focused on the keyboard, and issues
of throughput are eliminated as the system is designed for one user at a time.


Bei
ng of musical nature, auditory displays are essential to the system. We chose
closed
-
ear headphones for a level of privacy for the individual user, and to reduce noise
pollution for both the user and those outside the system (Sherman and Craig 172, 2003).
This reduction keeps the user from needing to raise the volume, so as long as the
headphones are good quality, there should be no issues of hearing damage. Once again,
the issues of encumbrance and throughput usually associated with head
phones are moot
wit
h our system.


Interaction Methods


The user will interact with the system through the piano keys and the mouse. In
certain sections of the program, the piano keys can be used to select options of what to
do next. These keys are clearly labelled on the hea
d
-
based display only when the options
are available to keep the ‘storyline’ very simple. Interacting with the mouse will mimic the
way the tool is used on a computer, using the standard curser as a selection tool
Jones and St. Jacques


28

(pointer). It is used to navigate the menu
screen, which is brought up by the user clicking
the mouse once when in ‘game play’ mode. The mouse and pointer and then used to
select options or songs. This method of interaction is familiar and effective.


D: Second Evaluation

Evaluation Process

1.

Subject

fills out pre
-
test questionnaire

2.

Interviewer explains the system’s physical setup aided by diagrams
. This mimics
the way the user would experience the system (as it is user
-
controlled, and based
on self
-
directed learning), and tests to see if the user wou
ld be able to understand
the system if participating in the real scenario.

3.

Interviewer presents slides (combinations of images and animations) of the test
sequence in order (see below). User selects options and moves through the test
without any assistance
/explanations by the interviewer.


Jones and St. Jacques


29



Jones and St. Jacques


30


4.

User fills out post
-
test questionnaire
, and is thanked for their time


Jones and St. Jacques


31

E: Questionnaires from 2
nd

Prototype

Pre
-
Test Questionnaire:


Jones and St. Jacques


32

Post
-
Test Questionnaire


Jones and St. Jacques


33

F: Questionnaire Results



Questions:

Subject 5

Subject 6

Subject 7

Subject 8

PRE
-
TEST QUESTIONNAIRE RESULTS

Age

29

32

31

46

Sex

M

M

F

F

Have you taken piano lessons at
any time?

Y
-
G2

N

Y
-
G4

Y
-
G2

How would you rate your current
ability to play music (
Any

melodic
instrument)?

3/10

2/10

4/10

3/10

How would you rate your current
ability to play
Piano
?

3/10

2/10

4/10

3/10

How would you rate your
knowledge of music theory?

2/10

1/10

4/10

2/10

I consider myself to be proficient
or a fast learner when exposed to
new technologies.

4/4

3/4

4/4


3/4

POST
-
TEST QUESTIONNAIRE RESULTS

I would feel comfortable taking
lessons from a piano teacher

4/4

4/4

3/4

3/4

The test sequence was intuitive
and easy to understand

3/4

2/4

4/4

4/4

I would prefer to see my
percentage displayed rather than
have the
program simply stop the
test

2/4

3/4

3/4

4/4

I would want to skip these tests
and simply go at my own level.

1/4

1/4

3/4

2/4

This device would encourage me to
play piano.


4/4

2/4

3/4

2/4

What should be
changed or added to
make this experience
better?

Auditory
instructions
would help

N/I

More feedback on
testing (suggestions,
tips, left hand weaker
than right hand)

Audio
feedback/readin
g the instruction
allowed

Additional comments

I like it… it’s a
good idea

It’s hard to tell
how intuitive the
test is
without
actually going
through it.

This would be
interesting to expand to
a take home device. I
would also like to be
able to download songs
I want to learn

N/I

Jones and St. Jacques


34



0
1
2
3
4
5
6
7
8
9
10
Subject 5
Subject 6
Subject 7
Subject 8
How would you rate your current
Ability to play
Piano
?

0
1
2
3
4
5
6
7
8
9
10
Subject 5
Subject 6
Subject 7
Subject 8
How would you rate your current
ability to play music (
Any

melodic
instrument)?


Jones and St. Jacques


35

Bibliography

Billings, Diane. “Teaching Learners from Varied Generations.”
Jour
nal of

Continuing
Education in N
ursing

35.3 (2004). 104. 19 Mar. 2012.
<
http://simplelink.library.utoronto.ca/url.cfm/256050
>.


Borsella, Marlene. Personal Interview. 15 Feb. 2012.


Bryce, Dave and Stephen Fortner. “From the Forums: Sight
-
Reading and Beyond.”
Keyboard

33.2 (2007). 15. 23 Feb. 2012.

<
http://simplelink.library.utoronto.ca/url.cfm/239626
>.


Cheng, Li
-
Te and John Robinson. “Personal Contextual Awareness through Visual Focus.”
IEEE Intelligent Systems
. 15 Feb. 2012.
<
http://wearcam.org/ieeeis/wearable_ai_sample_article.pdf
>.


Cooper, Thelma. “Adults’ Perceptions of Piano Study: Achievements and Experiences.”
Journal of Research in Music Education

49.2 (2001). 156
-
168. 19 Feb. 2012.
<
http://simplelink.library.utoronto.c
a/url.cfm/256207
>.


“Copyright Act”. The Department of Justice Canada (2005). 29 Feb. 2012. <
http://laws
-
lois.justice.gc.ca/eng/acts/C
-
42/FullText.html
>.


Finale. “Features: Entering Notes.” finalemusic.com (2012). 29 Feb. 2012.
<http://www.finalemusic.com
/PrintMusic/Features/EnteringNotes/default.aspx>.


Jutras, Peter. “The Benefits of Adult Piano Study as Self
-
Reported by Selected Adult Piano
Students.”
Journal of Music Education

54.2 (2006). 21 Mar. 2012.
<
http://simplelink.library.utoronto.ca/url.cfm/25
8243
>.


Sherman, William R. and Alan B. Craig.
Understanding Virtual Reality
. San Francisco:
Morgan Kaufmann Publishers, 2003.


Wristen, Brenda. “Cognition and Motor Execution in Piano Sight
-
Reading: A Review of
Literature.”
Applications of Research in Mus
ic Education
24.1 (2005). 44
-
56. 15 Feb
2012. <
http://simplelink.library.utoronto.ca/url.cfm/245758
>.


Zhou, Yu, Yu, Yao, Wang Ziqiang and Sidan Du. “
Piano AR: A Markerless Augmented
Reality Based Piano Teaching System
.”
Third International Conference on I
ntelligent
Human
-
Machine Systems and Cybernetics

(2011). 15 Feb 2012.
<
http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=06038212&tag=1
>.


“20 Best Applications for iPhone.” iPhoneNess.com (2010). 15 Feb. 2012.
<
http://www.iphoneness.com/iphone
-
apps/20
-
best
-
piano
-
applications
-
for
-
iphone/
>.