Symbolic Encoding of Neural Networks using
Communicating Automata with Applications to
Verification of Neural Network Based Controllers*
Li Su, Howard Bowman and Brad Wyble
Centre for Cognitive Neuroscience and Cognitive Systems,
University of Kent,
Canterbury, Kent, CT2 7NF, UK
{ls68,hb5,bw5}@kent.ac.uk
*To Appear in Neural

Symbolic Learning and Reasoning Workshop at Nineteenth International
Joint Conference on Artificial Intelligence, EDINBURGH, SCOTLAND, 2005.
Outline
•
Background:
–
Symbolic Computation
–
Sub

symbolic Computation
•
Motivation
for integrating Symbolic and Sub

symbolic
Computation
–
Cognitive Viewpoint
–
Application Viewpoint
•
Formal Methods
–
Model Checking
–
Specification
–
Properties
–
Result
•
Summary
Background 1: Symbolic Computation
•
Traditional symbolic computation:
–
Systems have explicit elements that correspond to symbols
organised in systematic ways, representing information in the
external world.
–
Programmes or rules can manipulate these symbolic
representations.
–
Key characteristic:
symbol manipulation
.
Background 2: Sub

symbolic Computation
•
Connectionism/neural networks are computational models
inspired by neuron physiology, which can be regarded as
sub

symbolic computation:
–
Aims at
massively parallel
simple and uniform processing
elements, which are interconnected.
–
Representations are
distributed
throughout processing elements.
Motivation 1: Cognitive Viewpoint
•
It has been argued that cognition/mind can be regarded as
symbolic computation. (E.g. SOAR, ACT

R and EPIC)
•
Sub

symbolic (neural network) architectures constitute
abstract model of the human brain.
Motivation 1: Cognitive Viewpoint (cont.)
•
Combining symbolic and sub

symbolic techniques to
specify and justify behaviour of complex cognitive
architectures in an
abstract
and
suitable
form.
–
Concurrent, Distributed Control, Hierarchical Decomposition
–
How do high

level cognitive properties
emerge
from interactions
between low

level neuron components?
•
Our approach is to encode and reason about cognitive
systems or neural networks in
symbolic
form.
–
E.g. Formal Methods.
–
Automatic mathematical analysis can be applied.
Motivation 2: Application Viewpoint
•
Connectionist networks can be applied to extending
traditional controllers in order to handle:
–
Catastrophic changes
–
Gradual degradation
–
Complex and highly non

linear systems
–
E.g. aircraft, spacecraft or robots
•
Reliability/Stability of adaptive systems (neural networks)
needs to be guaranteed in safety/mission critical domains.
•
However, connectionist models rarely provide an
indication of the
accuracy
or
reliability
of their predictions.
Formal Methods: Model Checking
•
Automatic analysis technique, which can be applied at
system
design stage
.
•
Checking whether a formal specification satisfies a set of
properties, which are expressed in a requirements language.
Model Checker
Inputs:
Yes +Witness / No + Counter

example
specification
properties
Output(s):
An Example of a Neural Network
Specification
I1
I2
H1
H2
O1
Environment
Tester
Input
Layer
Hidden
Layer
Output
Layer
NeuralNet
Note: this is not a realistic model of controller, but a “
toy
” model to
evaluate the ability of model checking neural networks.
Neuron Automaton
Input
Middle
Output
k
: identify of neuron;
t
: local clock;
: activation of neuron
i
;
i
: pre

synaptic neuron identity;
: speed of update;
: activation of neuron
k
;
j
: post

synaptic neuron identity;
: sigmoid function;
: error;
: net input;
: weight;
: learning rate.
Requirements Language
Requirements Language (cont.)
•
Reachability Properties:
–
E.g.
•
Safety Properties:
–
E.g.
–
•
Liveness Properties:
–
E.g.
Note: the state formula
success
is
true
when
SSE
is less than a specified value.
Result
•
The network satisfies the following properties and is
guaranteed to learn XOR according to the required timing
constraints using BP learning. It also guarantees the
learning process is eventually stabilised.
–
–
deadline
success
……
Summary
•
Formal methods are justifiable techniques to represent low

level neural networks. They can also help to understand
how high

level cognitive properties
emerge
from
interactions between low

level neuron components.
•
Formal methods may allow neural networks within
engineering applications to be specified and justified at the
system
design stage
.
•
Verifications may give theoretically well

founded ways to
evaluate and justify learning methods. Some p
properties
can be hard to justify by simulation.
–
Simulations can only test that something occurs, but are unable to
test that something can
never
occur without
explicit
mathematical
analysis. (An open issue.)
Comments 0
Log in to post a comment