visual collision detection

apricotpigletΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 4 χρόνια και 2 μήνες)

90 εμφανίσεις

Insect neural networks as a
visual collision detection
mechanism in automotive
situations


Richard Stafford (1), Matthias S. Keil
(2), Shigang Yue (1), Jorge Cuadri
-
Carvajo (2), F. Claire Rind (1)


1) School of Biology, Ridley Building, University of
Newcastle upon Tyne, NE1 7RU, UK.

2) Instituto de Microelectronica de Sevilla (IMSE), Centro
Nacional de Microelectronica (CNM),

Avda. Reina Mercedes, 41012, Sevilla, Spain

Structure of the talk


Introduction


Improvements to the LGMD model


Light on and light off pathways


Testing the LGMD model


Methods and Results


Further improvements


Biologically inspired filtering of images by lateral
inhibition


Analysing the filtered images using EMD like
structures


Conclusions

Locusts as collision detectors


The Lobula Giant Movement Detector
(LGMD) neuron responds most
vigorously when objects of certain
speeds and sizes approach, as if on a
direct collision course


This has been linked to a predator
avoidance, gliding behaviour in flying
locusts


Predator avoidance caused by the
LGMD

Angular subtense of object

LGMD Spikes

Inputs and structure of the LGMD

Why use the Locust LGMD to
detect automotive collisions?


Evolutionary honed collision avoidance
system


Efficient circuit


based on insect
neurons


Neural architecture well studied


Responds optimally to imminent
collisions


Simulated networks respond in a
similar manner to real locust

Limitations of existing model

(e.g. Rind and Bramwell, 1996;
Blanchard et al., 2000)


Simulations only tested in simple closed
environment


Model needs to work in real automotive
situations


Biology of the LGMD is not fully used


model only responds to change in light

Structure of the talk


Introduction


Improvements to the LGMD model


Light on and light off pathways


Testing the LGMD model


Methods and Results


Further improvements


Biologically inspired filtering of images by lateral
inhibition


Analysing the filtered images using EMD like
structures


Conclusions

Model Improvements


Light on
and Light off Pathways


Small scale spatial antagonism
between the pathways helps
eliminate noise caused by vibration
etc.


Larger scale antagonism can interfere
with collision alerts


Model Improvements


Light on and
Light off Pathways and Block Sum
Cells

Input Image

‘S’ units



light on ~ light
off

Block Sum Cells

Allow small scale

antagonism of

pathways only

Location of BSC in model

Block sum cells

occur here

Model Improvements
-

Block Sum
Cells

Sum light on (
-
ve) and

light off (+ve) excitation

to obtain net excitation

Excitation

(+ve only)

is passed

to the

LGMD

from the

BSC

Block sum cells obtain

excitation from a 10x10

section of the array of

‘S’ units

Light on and

Light off excitation

from ‘S’ units

Block Sum Cells

LGMD

Structure of the talk


Introduction


Improvements to the LGMD model


Light on and light off pathways


Testing the LGMD model


Methods and Results


Further improvements


Biologically inspired filtering of images by lateral
inhibition


Analysing the filtered images using EMD like
structures


Conclusions

Testing the model in automotive
situations

Input video

sequences

8


25 Hz


Input via frame
-

grabber of

Playstation

images

8.3 Hz

Detecting collisions


Membrane
potential of LGMD
is obtained from
sum of BSC


If a threshold is
exceeded then the
LGMD produces
spikes


If
>

2 spikes in 3
timesteps then
collision detected

Results: LGMD model

Results show % of times collision was detected even if no collision occurred

Stationary car

100 %

Moving Car

90 %

Head on with

moving Car

100 %

Entering Tunnel

0 %

General Driving

0 %

Driving in close

proximity

0 %

Translating cars

70 %

Why do translating cars prove

problematic?



Excitation is much higher in the LGMD for translating objects



Locust LGMD ignores translating objects partially due to


differences in mathematics of object approach

Structure of the talk


Introduction


Improvements to the LGMD model


Light on and light off pathways


Testing the LGMD model


Methods and Results


Further improvements


Biologically inspired filtering of images by
lateral inhibition


Analysing the filtered images using EMD like
structures


Conclusions

Image Filtering by LGMD network


‘S’ units
only excited
by objects
moving in
close
proximity to
car


e.g.


Colliding


or
translating
objects

No threat

Threat

Input Image

‘S’ units

Analysing the biologically filtered
images


Analysing patterns of excitation in ‘S’ or
‘BSC’ layers over time shows:


No or little excitation


no threat. LGMD does
not reach threshold


Excitation moving in one direction over time


no threat, translating object. LGMD spikes can
be suppressed


Excitation moving in all directions over time


collision threat, object on collision course is
expanding in all directions. LGMD spikes and
produces collision mitigation response


Structure of the talk


Introduction


Improvements to the LGMD model


Light on and light off pathways


Testing the LGMD model


Methods and Results


Further improvements


Biologically inspired filtering of images by lateral
inhibition


Analysing the filtered images using EMD
like structures


Conclusions

Incorporation of simple Elementary
Movement Detector like units
(EMDs) into the model


EMD like units take input from the Block
Sum Cells


simplified visual environment


One detected ‘Right’ movement and one
‘Left’ movement


If membrane potential of ‘left’ EMDs was


> 5 x potential of ‘right’ EMDs at time t or
time t
-
1 then LGMD spikes were
suppressed for time t, t+1 & t+2

Location of EMD like units

BSC

EMDs

Suppression

of LGMD spikes

Results: LGMD incorporating EMDs

Results show % of times collision was detected even if no collision occurred

Stationary car

85 %

Was 100 %

Moving Car

80 %

Was 100 %

Head on with

moving Car

50 %

Was 100 %

Entering Tunnel

0 %

Unchanged

General Driving

0 %

Unchanged

Driving in close

proximity

0 %

Unchanged

Translating cars

20 %

Was 70 %

Results: LGMD and EMDs


Incorporation of EMDs reduce false collision
alerts


Real collision detection was also reduced


EMD model was very simple. Using a more
advanced (adaptive) model may improve
the responses


Non bio
-
inspired image analysis could also
be used on the biologically filtered ‘S’ units
to improve model performance

Conclusions


Locust based LGMD model can be used for
automotive collision detection


In some situations modifications are
needed as the LGMD’s function in
automotive situations is quite different to
the evolved function in the locust


The biologically filtered image can be
analysed to further assess the threat of
collision

Acknowledgements


Project funded by Future and
Emerging Technologies Grant from
European Union


(LOCUST


IST
-

2
002
-
38097)


We would like to thank Marrti
Soininen of Volvo Car Corporation for
supplying the video footage of car
crashes

Other Improvements to the LGMD
model


On
-
Off cells look at absolute change in
image


Lateral inhibition has a greater potential
spread to eliminate more non threatening
situations


Spiking threshold of LGMD is self variable
to allow a greater range of visually complex
scenes to be investigated


Model parameters tuned using a Genetic
Algorithm to automotive situations

Differences between automotive
collisions and predator avoidance in
locusts


Locusts respond to
small, fast moving
predators


Final excitation, just
before predator
strikes, is much higher


This can be used to
distinguish between
different object types


Small translating
objects produce less
excitation than larger
objects