Introduction to computational intelligence

strawberrycokevilleAI and Robotics

Nov 7, 2013 (3 years and 9 months ago)

100 views

1
Introduction to computational intelligence
Computational intelligence:
definition, future, challenges
Igor Farkaš
Centre for Cognitive Science
DAI FMFI Comenius University in Bratislava
Based mainly on: Duch W. (2007): Towards comprehensive foundations of computational
intelligence. Lecture Notes in Computer Science, Springer, Vol. 63, 261-316.
2
Key concepts of CI

Rely on numerical data, rather than “knowledge”

Pattern recognition
(Bezdek, 1994)

Adaptation
(Eberhart)

Neural, evolutionary and fuzzy systems together
(Fogel,
1995)

Study of the design of intelligent agents
(Poole, 1998)

Proposed definition: CI is a branch of computer science
studying problems for which there are
no effective
computational algorithms

(Duch, 2007).
3
Features of CI methods

various methods in CI

share the feature of being subsymbolic

data-driven, where

the structure (knowledge) emerges bottom-up

rather than being imposed from above (pre-wired)

directly draw on environment

Relationship to AI and ML?
AI


CI
ML
4
Towards foundations of CI

Low-level:
perception, sensory-motor coordination (NNs)

Intermediate level:
fuzzy concepts, associative reasoning

High level:
traditional GOFAI, probabilistic models

Integration of levels necessary in agents


crisis of the richness” (lots of methods available)

Artificial neural networks (MLPs, RBFs, RNNs)

Extended to graphical models

Fuzzy logic – generalization of multi-valued logics

When merged: neuro-fuzzy systems

Evolutionary computing – for complex tasks
(Duch, 2007)
5
Challenges for CI


no free lunch” theorems
(Wolpert and Macready, 1997)

create flexible systems that can configure themselves

humans are very flexible in finding alternative ways
to solve a given problem

meta-learning research – ability to go beyond
parameter optimization (multiple-expert systems)

Computing and cognition as compression

dimensionality-reducing techniques
6
Role of semantics

syntax vs. semantics

Including semantics into the system

Symbolic s
.
– requires an interpreter (symbol grounding problem)

can be added by designer (knowledge base)

stable models, truth values (in a formal system)

Subsymbolic s
.
– does not (representations are grounded)

How to build it? (interaction with environment)
7
Meta-learning

via search in the model space

requires detailed characterization of the space of
possible models

in ANNs, theoretical results (universal approximation
property) do not guarantee efficient solutions

non-homogeneous
adaptive systems
desirable


homogeneity” of current approaches in terms of

uniform feature space

involved models
8
Similarity-based framework for meta-learning

chosen metrics is important

Various norms (L1, L2), cosine, scalar product

Using kernels for high-dim. feature spaces

Multiple transformation steps – layered graphical models

Primary goal: discover hierarchical sets of features

hierarchical models developed

MLPs and RBFs – as implementations of sets of
hierarchical fuzzy threshold logic gates
9
Transformation-based CI theory

Enhancements of feature extraction (from a small array of
sources)

separate basic functions and receptive fields for each dimension

scaling the features

creating new features via linear combinations

Extracting information from all features

expansion of the feature space (kernel methods)

increasing data dimension facilitates data separation
10
Prototype-based rules for data understanding

extraction of logical rules in most machine learning

Problems: many 'natural' categories are not natural

inspiration from human cognition:

human categorization based on memorization of exemplars and
prototypes

combining feature selection and prototype selection
methods useful

Committees of models (
mixture of experts
)
11
Important concepts from computational
learning theory

CLT analyzes the sample complexity an computational
complexity of inductive learning.

trade-off b/w

expressiveness

of hypothesis language and

ease

of learning


bias-variance trade-off

for model estimators

few parameters

h
igh bias, low variance

too many parameters

high variance, low bias

regularization

imposing constraints to avoid overfitting

PAC

(probably approximately correct) learning algorithms

stationarity

(of data) assumption
12
Open problems for CI

CI that goes beyond probabilistic pattern recognition

Existing methods cannot learn difficult problems

Problems requiring reasoning based on perceptions
should be explored

Methodology of evaluation and development of CI
methods needed

Cognitive robotics may be an ultimate goal