New technologies supporting people

spectacularscarecrowΤεχνίτη Νοημοσύνη και Ρομποτική

17 Νοε 2013 (πριν από 3 χρόνια και 9 μήνες)

51 εμφανίσεις

New technologies supporting people
with severe speech disorders


Mark Hawley

Barnsley District General Hospital and
University of Sheffield

Research Projects


STARDUST

(Speech Training and Recognition
for Disabled Users of Assistive Technology)


DoH NEAT programme



OLP

(Ortho Logo Paedia)


EU Vth Framework (Quality of life and
management of living resources)

Research Team


Department of Medical Physics and Clinical
Engineering, Barnsley District General Hospital

(Mark Hawley, Simon Brownsell)


Institute of General Practice and Primary Care,
University of Sheffield

(Pam Enderby, Mark Parker,
Rebecca Palmer)


Department of Computer Science, University of
Sheffield

(Phil Green, Nassos Hatzis, James Carmichael)


Dysarthria


A neurological motor speech impairment


characterised by slow, weak, imprecise and/or uncoordinated
movements of the speech musculature.


Frequently associated with other physical
disabilities


170/100 000
(Emerson & Enderby 1995)


Speech is often difficult to understand
(unintelligible) and variable (inconsistent)


Severe = <40% intelligible

Intelligibility and Consistency



Normal’ speech will be almost 100%
intelligible

and with few articulatory differences over time
(
consistent
).


‘Severe’ dysarthria may be completely
unintelligible

to the naïve listener and will show
high variability (
inconsistent
)


but may show
consistency

of key elements which will
make it more
intelligible

to the familiar listener.


STARDUST is concerned with
consistency


OLP with

intelligibility

STARDUST


To develop demonstrators of speech
-
driven
environmental control and voice output
communication devices for people with dysarthria



To develop a reliable small vocabulary speech
recogniser for dysarthric speakers



To develop a computer training program to help to
stabilise the speech of dysarthric speakers



Speech recognition systems


Speech
-
input writing programmes (eg Dragon)


large vocabulary, speaker adaptive



Speech
-
input control systems


small vocabulary, speaker dependent



Environmental control systems


speech
-
in
-

speech
-
out


Mobile phones


Smart homes


Speech
-
input writing programmes


Normal speech
-

with recognition training can get
>90% recognition rates
(Rose and Galdo, 1999)


Dysarthric speech
-

mild 10
-
15% lower recognition rates
(Ferrier, 1992
)


Declining as speech deteriorates
-

by 30
-
50% for single
words
(Thomas
-
Stonell, 1998, Hawley 2002)

Performance of a commercial speech
-
recognition environmental controller

(in ‘ideal’ conditions)

Recognition rate
N=6
Dysarthric subject 1
60%
Dysarthric subject 2
80%
‘Normal’ control
98%
STARDUST


To develop demonstrators of speech
-
driven
environmental control and voice output
communication devices for people with dysarthria



To develop a reliable small vocabulary speech
recogniser for dysarthric speakers



To develop a computer training program to help to
stabilise the speech of dysarthric speakers



Recognition technology


Small vocabulary


Speaker dependent


uses hidden Markov models


based on HTK
(University of Cambridge)

Recognition increases with amount of training data

40
45
50
55
60
65
70
75
10
16
22
28
34
40
46
52
58
64
70
76
82
% of training
Recognition %
3
5
7
9
11
13
15
STARDUST recogniser performance


(N=number of words used for training)

Intelligibility
Single words-
sentences
STARDUST
recogniser
N=6
STARDUST
recogniser
N=20
STARDUST
recogniser
N=28
Commercial
speech
recognition
ECS
N=6
Subject 1
0% - 0%
64%
80%
85%
60%
Subject 2
22% - 34%
100%
-
-
80%
Control
100%
100%
100%
100%
98%
STARDUST


To develop demonstrators of speech
-
driven
environmental control and voice output
communication devices for people with dysarthria


To develop a reliable small vocabulary speech
recogniser for dysarthric speakers



To develop a computer training program to help to
stabilise the speech of dysarthric speakers

(ie improve consistency)


Training tool


Quantitative and qualitative real
-
time visual
feedback to improve consistency


at word level


at sub
-
word level


Can be used by the client alone or with
carer or therapist


Training tool records examples of words
-

used to build recogniser


Consistency measure

word level

-56
-55
-54
-53
-52
-51
-50
-49
-48
1
2
3
4
5
6
7
8
9
10
11
mixture #
consistency
Mixtu
re #
# normal
utterances
of each
word
#
dysarthric
utterances
of each
word
1
10
0
2
9
1
3
8
2
4
7
3
5
6
4
6
5
5
7
4
6
8
3
7
9
2
8
10
1
9
11
0
10
OLT visual feedback (sub
-
word)

(Hatzis and Green 1999)

STARDUST
-

conclusions


Recogniser that recognises severely dysarthric
speech
-

well but not perfectly


next step to test in real usage


Computer
-
based training program to improve
consistency


word level and sub
-
word level in future


collects lots of speech data for recogniser


Develop demonstrators of environmental
controller and speech
-
output device

OLP


To supplement speech therapy by providing
computer
-
based tools for audio
-
visual feedback to
improve clients’ speech production



intelligibility

as well as
consistency



To make this available remotely using distance
learning techniques

Partners


Institute for Language and Speech Processing,
Greece


University of Sheffield, UK


Royal Institute of Technology, Sweden


Polytechnic University of Madrid, Spain


ARCHES, France


Unisoft Software Applications SA, Greece


Logos Centre fo Speech
-
Voice Pathology, Greece


Barnsley District General Hospital, UK

Client groups


Dysarthria


Pre
-
lingual and severe hearing impairment


Cleft lip and palate and velopharyngeal
incompetence

Changing speech patterns


Accurate and consistent feedback


Provide target speech patterns and feed
back deviations


Repeated practice



Drawbacks of conventional therapy


feedback may become inconsistent


lack of time leads to lack of practice

OLT visual feedback (sub
-
word)

Current systems


IBM Speech Viewer


Indiana Speech Training Aid


Video Voice Speech Training System


Speech Rehabilitation Speech Training Aid
for the Hearing Impaired (HARP)


Desirable features


should provide a contrastive visual training
ie the correct model of a reference speaker and the deviant
production of the client should be shown simultaneously to
be compared with each other



the visual pattern must be


attractive


easily comprehensible


shown without delay

OLP features


Ability to contrast desired target and
undesirable utterances (on same display)


Mapping of articulatory information to 2D
visual display in real
-
time


Motivating displays and exercises


Flexibility
-

can be individualised by
therapist


Can be used at home by client