Neural Networks and Statistics:

chickenchairwomanAI and Robotics

Oct 19, 2013 (3 years and 7 months ago)



Neural Networks and Statistics:

Intelligence and the Self

Prof Bruce Curry and Dr Peter Morgan

Cardiff Business School, UK



Generally seen as part of the discipline of AI, although to us they are
statistical devices.

They originated as models of the workings of the brain

‘A Logical Calculus of Ideas Immanent in Nervous Activity’

This implies they are an alternative to symbolic computation, eg. rule based

Rules and chaining methods, inference through formal logic.

NNs carry out sub
symbolic computation

They copy the physical workings of the brain

Neurons + connections



Some Details

Feedforward networks

Inputs, Outputs, Hidden Nodes

Hidden Node activation: step functions or sigmoids.
These are fuzzified steps

Hidden nodes provide the intelligence we seek to model

They are like specialised parts of the brain brought into
action when needed

They can describe Fuzzy IF/THEN rules.

Hence we don’t need a strict distinction between NNs
and rule based methods.


NN learning

Learning is an essential component of intelligence

NN’s can learn rules

Learning through weight adjustment and optimisation

Minimum RMS (equivalent to least squares)

Universal Approximation through use of a flexible functional form



Hence in statistical terms we have a form
of nonlinear regression

The network learns the underlying shape
describing the relationship between Y and

Example, Mexican Hat


NNs , Statistics and Intelligence

Hence NNs are statistical devices

Supervised and unsupervised learning

NNs provide nonlinear alternatives to standard
statistical methods

They are applied to traditional statistical tasks



Statistics, in its aims, is almost antithetical to intelligence

The aim is automated judgment based on standard

This is achieved by standard numerical measures, for
example of the strength of a conclusion

The prime example is testing hypotheses, where we
have a significance level or ‘p value’

Everyone is supposed to agree that for example

p = .001 gives a strong conclusion, with in fact less than
1% chance of being wrong

This is in practice rather idealistic!!



(a) The nature of the Self and the Mind Body Problem.

This includes issues involving consciousness.

(b) The nature of Intelligence and the limits of Strong AI

AI texts generally insist that the subject is concerned with computer models
which operate in a way we would perceive as being intelligent. This has
implications for the Turing test. A simple version of the test involving just
outward behaviour or evidence would not be sufficient for intelligence.
Interestingly, the attacks by Searle (through his ‘Chinese room’) on ‘Strong
AI’ or ‘Computationalism’ have a similar flavour. It has been argued that a
connectionist approach provides an escape from the problem.


(c) Gödel’s theorems and their implications for
AI. The theorems serve to identify a difference
between truth and provability, in which case
machine intelligence inevitably has limitations. It
has been argued that a system which can learn,
for example a Neural Network, can escape from
the implications of the Gödel theorems. However,
because the difficulties arise specifically with
referential statements in formal systems this
is unlikely to be true.



Our networks are detailed numeric models, which although based
on some sophisticated methods are still quite simple and

They find optimal weight values through blind search

They learn through weight adjustment

They are statistical devices they also don’t embody intelligence

They can be analysed using the standard tools of applied

They are too detailed and too primitive to be regarded as models of
intelligence as such ( as opposed to the underlying processes
which produce intelligence)

However, we can build specialised networks to model for example


The Self

A Self is not necessary for intelligence, but it is for ‘higher’ levels of

E.g. My cat is intelligent, an ant colony displays aspects of intelligence>
neither has a self

The self is the driver of the brain, but where and what is it?

If we physically reverse engineer a brain, in such a way that the copy
functions exactly as the original, where is the self?

This is in fact a connectionist approach

What happens with the self and NN’s?

Our types of network are basic low level number processors, which can in
fact be given higher level interpretations

Can in fact simulate consciousness ( the self) in more specialised networks

This is not ‘real’ but simulated consciousness. There can be a model of the
self and consciousness of the self.

Can we simulate a person who controls his/her self?


Searle Argument: Chinese Room

computation view:

passing Turing test does not necessarily imply intelligence ( test
based on the end result not the process)

Chinese language translation is just a mechanical linking of symbols,
with syntactic rules. Maybe like an electronic dictionary. What about
a human interpreter: does he/she operate mechanically?

Symbols have no internal meaning, so AI is not attainable. (There is
no self, to achieve understanding).

But, we can simulate some of the process of human reasoning, eg.
using Prolog.

Partial simulation/ emulation of intelligence is attainable, even if the
mind is not simply a digital computer

But we must pass the Turing test in a way regarded as being


NNs, Connectionism and Searle?

As above, our own networks are just mechanical
devices, without intelligence

Connectionist answer to Searle? Intelligence as
manifested in the brain consists of symbol
manipulation plus lower level ( sub symbolic)
processes). Therefore AI is possible.

But, refer back to point above about perfect
physical replica of the brain. Where in this
perfect replica is intelligence and the self?


GOEDEL theorems

These show distinction between truth +

Hence incompleteness of any system of axioms

Only arises with self referential statements: ‘This
sentence is unprovable’

Show limitations of computationalist approach,
but don’t destroy it. Can still simulate/emulate
human thought processes to some degree


This is because it shows limitations of human

Maths and reasoning are about derivation from
initial axioms, therefore the truth of those axioms
is unprovable. In this sense the Goedel
theorems don’t do much damage.

Theoretically possible to have computational
intelligence which has proofs but also

Also, Goedel theorems deal with ‘perfect’
reasoning: human reasoning may be imperfect
and this may be simulated.


Importance for NNs?

Can’t argue that learning allows an escape: but
learning of non self
referential statements is OK

What happens with NN’s and self learning?

Can simulate consciousness in the network, by
having specialist nodes

but isn’t this still limited by Goedel?

Goedel theorems help with AI; they show that all
reasoning is subject to limitations and axioms
need to be ‘imported’.