Feature Creation and Concept Learning empirical evidence and ...

apricotpigletAI and Robotics

Oct 19, 2013 (3 years and 9 months ago)

89 views

1

Feature Creation

and Concept Learning:

Empirical Evidence

and Neural Modeling

Michael Fink
Hebrew Univ. Jerusalem

Gershon Ben
-
Shakhar
Hebrew Univ. Jerusalem

David Horn
Tel Aviv Univ.

2

Introduction

How do we bridge the gap between
available low
-
level features and the
high
-
level features needed in a
complex environment?


We

manipulate

complex

perceptual

concepts

with

speed


We

learn

new

concepts

from

single

exposures

to

exemplars


These

characteristics

require

complex

features


Feature

creation

capabilities

are

therefore

a

necessity





(Schyns,

Goldstone

&

Thibaut,

1998
)

3

Purpose of this work

Demonstrate experimentally that, while learning new
concepts, features are being automatically created. These
features are intermediate structures, composites of the
elementary inputs.

Concepts are composites of features. Features are shared
by different concepts.


Show analog behavior of a neural network mimicking the
experiment.

4

Experiment
1

Use a setup of 8 colored cubes to define four
concepts (combinations of 4 cubes each) and
demonstrate emergence of four features (2 cubes
each). The experiment was carried out on 27
subjects. Training sessions continued until perfect
learning was reached.

5

Experiment 1


Materials


Learning session


Testing session


Results

8
binary inputs lead to
2
**
8
configurations.

Different colors determine uniqueness of each cube. This
allows presentation of the figure from different spatial
perspectives, thus eliminating bias of representation.

6

Experiment
1


Materials


Learning session


Testing session


Results

7

Experiment 1


Materials


Learning session


Testing session


Results

8

Experiment
1


Materials


Learning session


Testing session


Results

Test: recall colors of a
concept. Order of recall
should demonstrate the
acquired features.

9

Experiment
1


Materials


Learning session


Testing session


Results

10

Experiment
1


Materials


Learning session


Testing session


Results

Diagonal &
Incongruent

Adjacent &
Congruent

Adjacent &
Incongruent

11

Free recall of a concept


Black…
Black &
Blue

Just a minute...

Yellow

Yellow, Green

Together

With


Hum


I

think


Maybe with

12

Experiment
1


Materials


Learning session


Testing session


Results

13

Neural Network
Modeling

14

Neural Network Modeling


Feed Forward networks


Eight input elements (p)


Four hidden neurons (h)


Four output neurons (a)


Train by changing weights to
minimize the error (a
-
t)
2


Modeling Framework


Multi Layer Perceptron


Attentional Constraint


Model Prediction

15

Neural Network Modeling

2

4

6

8

1

2

3

4

Input

Hidden Neurons

Layer
1

1

2

3

4

1

2

3

4

Hidden Neurons

Output

Layer
2


Modeling Framework


Multi Layer Perceptron


Attentional Constraint


Model Prediction

16

2

4

6

8

1

2

3

4

Input

Hidden Neurons

Layer
1

1

2

3

4

1

2

3

4

Hidden Neurons

Output

Layer
2

Neural Network Modeling


Modeling Framework


Multi Layer Perceptron


Attentional Constraint


Model Prediction

change weights to
minimize the error

sum
i
(
a
i
-
t
i
)
2
+
w
2

17

Free recall of a concept


Black…
Black &
Blue

Just a minute...

Yellow

Yellow, Green

Together

With


Hum


I

think


Maybe with

18

Free recall of a concept

Modeling with the neural network by testing it under
sub
-
threshold conditions, mimicking mental search:
Winning Input Activation.

Start with zero input activations. Search for
ε

update
that maximally reduces error of desired output. Choose
winner and update only it. Similarly choose loser as one
who maximally increases error and deactivate it.

Proceed until unit activations reach
0.5
.

19

Free recall of a concept

20

2

4

6

8

1

2

3

4

Input

Hidden Neurons

Layer
1

1

2

3

4

1

2

3

4

5

Hidden Neurons

Output

Layer
2

Neural Network Modeling


Modeling Framework


Multi Layer Perceptron


Attentional Constraint


Model Prediction

Learn four concepts

21

Neural Network Modeling

Learn four concepts

Learn a
5
th concept in
limited

time

Congruent
condition

Incongruent
condition


Modeling Framework


Multi Layer Perceptron


Attentional Constraint


Model Prediction

22

2

4

6

8

1

2

3

4

Input

Hidden Neurons

Layer
1

1

2

3

4

1

2

3

4

5

Hidden Neurons

Output

Layer
2

Neural Network Modeling


Modeling Framework


Multi Layer Perceptron


Attentional Constraint


Model Prediction

Incongruent condition

23

Neural Network Modeling

Extracted features
facilitate future
concept learning

2

4

6

8

1

2

3

4

Input

Hidden Neurons

Layer
1

1

2

3

4

1

2

3

4

5

Hidden Neurons

Output

Layer
2


Modeling Framework


Multi Layer Perceptron


Attentional Constraint


Model Prediction

Congruent condition

24

Experiment
2

25

Experiment
2


Learning session


Testing session


Results

Learn four concepts

Congruent
Group

Incongruent
Group

Learn a
5
th concept in
limited time

Test the model

s
prediction

that learning future
concepts based on the
features previously
extracted will be
significantly facilitated

26

Experiment
2


Learning session


Testing session


Results

Learn a
5
th concept

Report
5
th concept

Congruent
Group

Incongruent
Group


White, Orange, Black & Blue


Learn four concepts

27

Experiment
2


Learning session


Testing session


Results

28

Summary


Purpose of this work was to demonstrate feature creation within the
learning process of new concepts.


Experiment
1
-

demonstrated feature creation. Features are shared
between different concepts.


Neural Network Modeling
-

description & prediction


Experiment
2
-

feature creation facilitates future learning.