Real-Time Feature Extraction and Classification of Prehensile EMG Signals

pancakesbootΤεχνίτη Νοημοσύνη και Ρομποτική

24 Νοε 2013 (πριν από 3 χρόνια και 10 μήνες)

171 εμφανίσεις

Real
-
Time Feature Extraction and
Classification of Prehensile EMG Signals

Master Thesis



Christopher Miller


Supervisor:
Marko Vuskovic



Department of Computer Science

San Diego State University

March 24, 2008

2

Copyright © 2008 by Christopher Miller

Agenda


Introduction


Electromyography (EMG) Signals


EMG Signal Processing


Classification


Experimental Results


Implementation


Conclusion


Future Work


Questions


3

Copyright © 2008 by Christopher Miller

Introduction

(1 of 2)


Numerous technological advances in prosthetic hands


Greater degrees of freedom


Continue to function as pincers

4

Copyright © 2008 by Christopher Miller

Introduction

(2 of 2)


The purpose of this thesis was to implement a
program that could perform real
-
time feature
extraction and classification of prehensile EMG
signals for the following grasps:

5

Copyright © 2008 by Christopher Miller

Electromyography (EMG) Signals:

Myoelectric Energy Detection


Motor units control groups
of muscle fibers


Brain recruits motor units to
innervate muscles for
movement


Myoelectric energy
produced as motor units
activate


Surface electrodes detect
myoelectric energy

6

Copyright © 2008 by Christopher Miller

Electromyography (EMG) Signals:

EMG Amplification


SENIAM Recommends


Pre
-
gelled Ag/Ag
-
Cl


Bipolar


0.8” inter
-
electrode distance


0.4” wide


Surface electrodes


Pre
-
gelled Ag/Ag
-
Cl


1” inter
-
electrode distance


0.875” wide


EMG amplification device


(Saksit Siriprayoonsak, 2005)


4 bipolar channels


1 reference channel

7

Copyright © 2008 by Christopher Miller

Electromyography (EMG) Signals:

Muscle Anatomy

8

Copyright © 2008 by Christopher Miller

Electromyography (EMG) Signals:

Mounted EMG Amplifier

9

Copyright © 2008 by Christopher Miller

EMG Signal Processing:

System Flowchart


Prosthetic

Hand
Controller

EMG
Amplification
Device

A/D
Converter


Classifier

Signal
Processing

Time Sample
Extraction

Feature
Extraction

Transformation

10

Copyright © 2008 by Christopher Miller

EMG Signal Processing:

Onset of Movement Detection



Lidierth (1986)


Hodges & Bui (1996)


Bonato
et al
. (1998)


Staude et al. (2001)


AGLRamp, AGLRstep

11

Copyright © 2008 by Christopher Miller

EMG Signal Processing:

Bonato Method







h
g
k
t
y
y
g
k
a
k
k
k






:
5,...

3,

1,
min
,
1
2
2
1
2
0

12

Copyright © 2008 by Christopher Miller

EMG Signal Processing:

Feature Extraction Methods


Mean Absolute Value (Hudgins
et al
., 1993)


Mean Absolute Value Slope (Hudgins
et al
., 1993)


Willison Amplitude (Willison, 1964)


Zero Crossings (Hudgins
et al
., 1993)


Slope Sign Changes (Farry
et al
., 1996)


Waveform Length (Farry
et al
., 1996)


Simple Square
-
Integral


Amplitude of the First Burst (Vuskovic
et al
., 2002)


Multiple Time Windows (Du
et al
., 2003)


Short
-
Time Fourier Transform


Wavelet Transform


Wavelet Packet Transform


Spectral Moments (Vuskovic
et al
., 2005)

13

Copyright © 2008 by Christopher Miller

EMG Signal Processing:

Feature Extraction Comparison

Method



Parameters



400 ms



300 ms



200 ms



MAV





96.11%



95.56%



93.89%



MAVSLP



I = 3



83.89%



81.67%



73.89%



VAR





90.56%



90.00%



86.67%



WAMP



H = 29



98.33%



95.56%



92.22%



Wave

form

L

ength





98.89%



98.89%



95.00%



Zero

-

Crossings



H = 28



96.11%



94.44%



83.89%



Slope Sign Changes



H = 100



96.67%



96.67%



86.67%



Squared Integral





90.56%



90.00%



86.67%



Spectral Moments



m = 1



K = 11



flag = 0



92.78%



89.44%



82.22%





14

Copyright © 2008 by Christopher Miller

EMG Signal Processing:

Feature Extraction Methods Employed (1 of 3)

Spectral Moments:




I
-
coefficients, calculated in advance:








1
1


π
2
]
[
)
(
N
N
k
k
f
j
SS
e
k
C
f
P











1
0
)
1
(

...,

2,


1,



],
[
]
[
1
]
[
k
N
i
SS
N
k
k
i
s
i
s
N
k
C




K
k
m
SS
m
SS
m
k
I
k
C
I
C
M
1
)
(
]
[
2
)
0
(
]
0
[
)
1
(
2
1
)
0
(
1



m
I
m
m




4
3
2
2
2
1
0
)
2
(
2
2
)
1
(
3
)
(

,
)
2
(
)
1
(
)
(

,
)
2
(
1
)
1
(
)
(

,
0
)
(
k
k
k
I
k
k
I
k
k
I
k
I
k
k
k
k



















...

3,

2,

1,


,
)
2
(
6
)
1
(
2
)
(
4
4




k
k
k
k
I
k
k










k
I
m
k
m
k
I
m
m
k
m
m
2
1
2
1
1
2
1
)
2
(
2
)
(













K
k
m
SS
m
m
m
k
I
k
C
M
I
M
R
1
0
)
(
]
[
2
)
0
(
2
15

Copyright © 2008 by Christopher Miller

EMG Signal Processing:

Feature Extraction Methods Employed (2 of 3)

Parameter

Range

Purpose

k

0..N

(N is sample size)

Time
-
lag parameter in I
-
coefficient and autocorrelation
function

m

0..4

Highest moment index to use in f
eature vector (i.e. m = 2
means that M
0
, M
1
, and M
2
are included)

Ts

0..N

Size of time sample to use for moment calculation

Flag

0..2

0

use straight moments

1

use M
0
and reduced moments for higher values of m

2

use strictly reduced moments


Spectral Moment Parameters

16

Copyright © 2008 by Christopher Miller

EMG Signal Processing:

Feature Extraction Methods Employed (3 of 3)

Waveform Length:






N
k
k
k
x
x
WL
2
1
17

Copyright © 2008 by Christopher Miller

Classification:

Methods


Artificial Neural Networks (McCulloch
-
Pitts, 1943)


ARTMAP Networks (Grossberg
et al
., 1976)


Mahalanobis
-
Distance Based ARTMAP Network
(Xu
et al
., 2003)


Maximum Likelihood Estimation (Fisher, 1912)


Mahalanobis Distance (Mahalanobis, 1936)

18

Copyright © 2008 by Christopher Miller

Classification:

Methods Employed


Maximum Likelihood Estimation (MLE)






Mahalanobis Distance













)
(
)
(
2
1
exp
)
2
(
1
)
|
(
1
2
1
C
C
T
C
C
x
S
x
S
x
p



)
(
)
(
)
(
1
x
S
x
x
MD
C
C
T
C










]
)
(
[
)
cov(
T
X
X
c
X
X
E
X
S






19

Copyright © 2008 by Christopher Miller

Experimental Results:

Impact of Log Transformation

20

Copyright © 2008 by Christopher Miller

Experimental Results:

Box
-
Cox Transformation


Box
-
Cox (1964):










0

,
log
0

,
1
)
(





y
y
y











0

),
log(
0

,
1
)
(
1
2
1
1
2
)
(
1







y
y
y
21

Copyright © 2008 by Christopher Miller

Experimental Results:

λ Optimization


Mahalanobis
-
distance Classifier using Spectral Moments


Flag = 0


K = 11


Ts = 250

22

Copyright © 2008 by Christopher Miller

Experimental Results:

Moment Optimization


Classifiers using Spectral Moments


Flag = 0


K = 11


Ts = 400


Lam = 0.06

23

Copyright © 2008 by Christopher Miller

Experimental Results:

Time Sample Optimization


Mahalanobis
-
distance Classifier using Spectral Moments


M = 2


Flag = 0


K = 11


Lam = 0.06

24

Copyright © 2008 by Christopher Miller

Experimental Results:

Channel Reduction (1 of 2)

25

Copyright © 2008 by Christopher Miller

Experimental Results:

Channel Reduction (2 of 2)


Mahalanobis
-
distance Classifier using Spectral Moments


M = 2


Flag = 0


K = 11


Lam = 0.06


3 channels

26

Copyright © 2008 by Christopher Miller

Experimental Results:

Feature Comparison (1 of 2)







Waveform Length
-




-

Spectral Moments

27

Copyright © 2008 by Christopher Miller

Experimental Results:

Feature Comparison (2 of 2)

MAH Classifier Performance at 264 ms Using Various Feature Vectors

Hybrid

Percent Used
for Testing

Spectral
Moments

Waveform
Length

M=0

M=1

M=2

10%:

99.34%

97.79%

98.54%

98.82%

98.51%

20%:

99.20%

97.83%

98.31%

98.27%

97.37%

30%:

98.84%

9
7.89%

97.90%

97.46%

95.45%

40%:

97.96%

97.70%

97.56%

96.20%

89.33%

50%:

96.30%

97.59%

96.92%

93.36%

26.07%

60%:

26.87%

97.37%

95.77%

25.88%

N/A

70%:

N/A

96.93%

89.73%

N/A

N/A

80%:

N/A

95.10%

25.78%

N/A

N/A

Average of
50% and below

98.33%

97.76%

97.85
%

96.82%

81.35%


28

Copyright © 2008 by Christopher Miller

Experimental Results:

Hybrid Approach

29

Copyright © 2008 by Christopher Miller

Experimental Results:

Spectral Moments


Best classification rate: 97.5%


Optimal time sample: 378 ms

30

Copyright © 2008 by Christopher Miller

Experimental Results:

Waveform Length


Best classification rate: 95%


Optimal time sample: 300 ms

31

Copyright © 2008 by Christopher Miller

Experimental Results:

Waveform Length Online Testing


Cross
-
validation


95% at 300 ms


Online validation


Sphere: 22/25 (88%)


Cylinder: 25/25 (100%)


Precision: 21/25 (84%)


Lateral: 17/25 (68%)


Total: 85%

32

Copyright © 2008 by Christopher Miller

Experimental Results:

Spectral Moments Online Testing


Cross
-
validation


89.4% at 355 ms


Online validation


Sphere: 24/25 (96%)


Cylinder: 25/25 (100%)


Precision: 25/25 (100%)


Lateral: 19/25 (76%)


Total: 93%

33

Copyright © 2008 by Christopher Miller

Implementation:

Program Files

Name

Type

Purpose

EMGProgGUIFC.m

GUI support functions for program

EMGProgGUIFC.fig

GUI front
-
end for program

Train.m

Function to generate classifier parameters S, C, N

GetClass.m

Function to classify data using current classifier

LOO
Validation.m

Leave One Out Validation procedure used for training

BonatoOnset.m

Function to determine onset of movement in recorded data

BonatoRealTime.m

Function to determine onset of movement in real
-
time

Features.m

Function to create feature vect
or using spectral moments

Moments.m

Function to compute spectral moments of time signal

GenerateI.m

Function to generate I coefficients for moment calculation

FeaturesWL.m

Function to create feature vector using waveform length feature

FeaturesMAV.m

Function to create feature vector from mean absolute value

BoxCox.m

Function to perform Box
-
Cox transformation

dflt_train.mat

MAT file containing classifier based on research data

EMGDLLmfile.m








MATLAB

DAQ functions MATLAB prototype file


EMGDLL.dll

D
ynamic Link Library (DLL) for DAQ card

DAQ_Functions.h

National Instruments DAQ functions header

DAQ_Functions.c


C

National Instruments DAQ functions implementation


34

Copyright © 2008 by Christopher Miller

Implementation:

Main Screen

35

Copyright © 2008 by Christopher Miller

Implementation:

Training (1 of 3)


36

Copyright © 2008 by Christopher Miller

Implementation:

Training (2 of 3)


Leave
-
one
-
out validation from 200
-
400 ms in 10 ms intervals


Optimal time explored at 1 ms
intervals


Both features scored based on
classification rate and time sample
required


Highest scoring feature selected and
system trained


37

Copyright © 2008 by Christopher Miller

Implementation:

Training (3 of 3)


May load training from files


Training recordings automatically stored with classifier


Training recordings saved to EMGRecordings folder

38

Copyright © 2008 by Christopher Miller

Implementation:

Real
-
Time Grasp Classification



Rest

Position



Start

Grasp



Hold

Grasp



Onset of movement

detected



Grasp

classified



Grasp tension

below threshold



Grasp tension

above threshold



Collect

grasp data



Onset of

movement not

detected



39

Copyright © 2008 by Christopher Miller

Implementation:

Real
-
Time Grasp Classification Screenshot

Implementation:

Real
-
Time Grasp Classification Video

41

Copyright © 2008 by Christopher Miller

42

Copyright © 2008 by Christopher Miller

Conclusion


Feasible approach demonstrated for real
-
time classification of
prehensile EMG signals


More natural approach reduces mental and physical demands
on operator


Only 3 EMG channels are necessary to classify the 4 grasps


Waveform Length proved to be valuable for small training sets


Spectral Moments enabled better performance (93% online)
with larger training sets and optimized parameters


K

= 11


m

= 2


flag

= 0


λ

= 0.06


Ts

optimized for training set

43

Copyright © 2008 by Christopher Miller

Future Work:

Additional Grasps


Previous research explored classifying 6 grasps, which
included small and large versions of balls and cylinders


Two
-
phased approach to classification is more likely to
succeed

44

Copyright © 2008 by Christopher Miller

Future Work:

Full Control


Channel 4 of current device can be employed for grasp control

45

Copyright © 2008 by Christopher Miller

Future Work:

Online Learning


Real
-
Time program should be written in multi
-
threading
capable language


Online learning capability with feedback from prosthetic hand


Mahalanobis
-
distance based ARTMAP network suggested


EMG Classifier

Library

(MATLAB
DLL

with
C
Interface
)

Deployed

Real
-
Time

Program

(
C++
or Java
)

Robot
ic
Hand
Controller

(C/C++)

EMG
Device

Drivers

(C)


Real
-
Time

EMG
Classifier
GUI

(
MATLAB
)

Train

classifier

Upload

classifier

Read EMG
Data

Classify
EMG Data

Update
grasp


Questions

47

Copyright © 2008 by Christopher Miller

Backup Slides

49

Copyright © 2008 by Christopher Miller

Differential Amplification