Brain Computer Interface as Sensor

Sh ya ma n t a

M.
Ha z a r i k a
,
Ad i t y

Sa i k i a
,
Si ma n t a

Bo r d o l o i
,
Uj j a l

Sh a r ma An d
Na ya n t a r a

Ko t o k y


De p a r t me n t Of Co mp u t e r Sc. & En g i n e e r i n g,
Te z p u r

Un i v e r s i t y

Te z p u r
, I n d i a

s h ya ma n t a @i e e e.o r g

Brain Computer Interface as Sensor
for Ambient Intelligent Living:

A Position Paper






Biomimetic and Cognitive Robotics @ TU


BCR@TU

conducts

research

in

the

area

of

Cognitive

Robotics

and

Knowledge

Representation

&

Reasoning
.

We

are

particularly

interested

in

Qualitative

Spatial

and

Temporal

Reasoning
.

This

translates

into

interest

in

Cognitive

Vision

and

Rehabilitation

Robotics
.



Our

research

within

Cognitive

Robotics

and

KR

&R

is

driven

by

biomimetics

i
.
e
.
,

examination

of

nature

particularly

human

intelligence

and

skills,

its

models,

systems,

processes,

and

elements

to

emulate

or

take

inspiration

from

these

designs

and

processes
.



For

development

of

prostheses

and

assistive

devices

within

Rehabilitation

Robotics

we

undertake

biomimetic

design
,

which

is

NOT

JUST

A

COPY

of

the

geometry!

For

us

biomimetic

design

is

biomimetic

geometry

together

with

functional

biomimesis
.


Brain Computer Interfaces


Brain computer interfaces


–
Use computers to sense human
thoughts and enable the users to
control external devices


–
Infer a user’s intentions using only
brain activity


–
Provide a non
-
muscular avenue for
communication


Applications

–
BCIs are aimed at assisting,
augmenting, or repairing human
cognitive or sensory
-
motor
functions.

e.g. locked
-
in syndrome
(cognitively unimpaired, but no motor
control)


•



Brain Computer Interfaces

•
Depending on application, BCI can be classified
as


•
Cognitive

•
Sensory

•
Motor


•
Motor BMI seeks to translate brain activity
from the central or peripheral nervous system
into useful commands to external devices.

•
Drive Prosthetics

•
Functional electrical stimulation


•
Motor BMI can be categorized as


•
Invasive

•
Partially Invasive

•
Non
-
Invasive

What is an EEG?

An

electroencephalogram

is

a

measure

of

the

brain's

voltage

fluctuations

as

detected

from

scalp

electrodes
.

It

is

an

approximation

of

the

cumulative

electrical

activity

of

neurons
.

•
Brain

–
set of interconnected modules

–
performs information processing
operations at various levels

•
sensory input analysis

•
memory storage and retrieval

•
reasoning

•
feelings

•
consciousness

•
Neurons

–
basic computational elements

–
heavily interconnected with other
neurons

Beta Rhythm

Alpha & Mu Rhythm

Grounding

Electrode Placement

Standard 10:20 System

Experiment Protocol

Bispectrum of EEG Signal

•
Bispectrum

is

the

expectation

of

three

frequencies
;

two

direct

frequency

components

and

the

third

the

conjugate

frequency

of

the

sum

of

those

two

frequencies
.


•
Knowing

the

Fourier

frequency

components

X(f)

the

bispectrum

B(f
1
,

f
2
)

can

be

estimated

using

the

Fourier
-
Stieltjes

representation
.


B(f
1
,

f
2
)

=

E(X(f
1
)X(f
2
)X*(f
1
+

f
2
))


Where

X*(f)

is

the

complex

conjugate

of

X(f)

and



E(

)

is

the

statistical

expectation

operator


Bispectrum Analysis

Bispectrum Analysis

•
Bispectrum

analysis

provide

a

way

to

evaluate

mental

representation

during

observation

and

imagination

of

hand

movement







•
Prior

visual

representation

of

motor

acts

make

difference

during

motor

imagination
.

Another experiment

•
Aim

is

to

classify

four

different

motor

imagery,

namely,

–
Both

Hands

Up

–
Tighten

Both

Fists

–
Left

Hand

Up

–
Right

Hand

Up

•
The

Protocol
















Start Audio Cue

Action Audio Cue

Stop Audio Cue

Relax and keep
your eyes closed.

Imagine
the action
.

End the task
and relax.

The Architecture

EEG Unit

Noise &
Amplitude
Normalization

Feature
Extraction
Unit

K
-
fold cross
validation

SVM

Filtration &
Normalization Unit

Motor
Imagery
Types

Classification Unit

Hybrid Features of
Bispectrum

•
Here we do not make use of the bispectrum feature
directly rather we use following two hybrid features of
bispectrum in order to retain the temporal as well as
frequency information within the EEG data.

•
Sum of Logarithmic Amplitudes (SLA)

to characterizes temporal
bispectral information.



θ

gives the principal domain.

•
First Order Spectral Moment (FOSM)

to characterizes frequency
information of the bispectrum.



N is the number of diagonal elements of Bispectrum


Figure : Bispectrum Estimation of the EEG Signals.

Top
-
left: left hand motor imagery; top
-
right: right hand motor imagery;

bottom
-
left: both hands motor imagery & bottom
-
right: both fists motor imagery.


Bispectrum Analysis

Classification

•
We have used RBF kernel
SVM for classification of
the MIs.

•
The original SVM algorithm
was proposed by Vladimir
Vapnik in 1970.

•
The result is cross
-
validated
through 10
-
Fold Cross
Validation.

Confusion Matrix


BCI Based Maze Game

With an aim of developing a non
-
invasive BCI to be used as an
intelligent assistive system, we have designed and developed a simple
maze game, where a player plays the game in real time by using his
brain signals.

Mapping of Motor Imageries with
Game Moves

Motor Imagery

Game Move

Both Hands Up

Move Forward

Right Hand Up

Move Right

Left Hand Up

Move Left

Tight Both Fists

Move Backward

What we have done so far?

BCI Integrated Collaborative Control

•
The idea is to integrate a BCI with a cognitive architecture for
collaborative control of a smart wheelchair.

•
The cognitive architecture mediates based on the extent of
automatic vs. manual control to be achieved.

•
AIM…

–
To help people with mobility disability (with or without cognitive
impairment) to achieve a level of independence so that carryout their
daily activities.

BCI Integrated Collaborative Control
Architecture

Automatic control
Module

Adaptation
Module

Mediator

Manual control
Module

Sensing

Control

Sensor Role

Actor Role

Brain Computer Interface

Assistive Device

Intelligent/Smart Wheelchair

BCI Integrated Collaborative Control
Architecture

•
Three layered control architecture
–

BCI; Superior
Control and Local Control.

•
The BCI plays a dual role that of an
actor as well as a
sensor
.

•
It not only does provide control commands to drive
the wheelchair but also monitor the
cognitive state
of
the user
-

his
confidence, cognitive workload and
wellbeing
, depending on which BCI could provide
assistance range from partial control of navigation to
complete autonomous mode .

Final Comments

•

Over the years AI has drifted away from its main
aim. This work is an attempt to focus on integrated
systems rather than component algorithms.

•

The cognitive systems paradigm needs to have its
source of ideas in human cognition. This position
paper describes work done at the Biomimetic and
Cognitive Robotics Lab at Tezpur University for
development of a BCI Integrated Collaborative
Controller for an intelligent wheelchair.