Introduction to computational intelligence

strawberrycokevilleIA et Robotique

7 nov. 2013 (il y a 7 années et 11 mois)

330 vue(s)

Introduction to computational intelligence
Computational intelligence:
definition, future, challenges
Igor Farkaš
Centre for Cognitive Science
DAI FMFI Comenius University in Bratislava
Based mainly on: Duch W. (2007): Towards comprehensive foundations of computational
intelligence. Lecture Notes in Computer Science, Springer, Vol. 63, 261-316.
Key concepts of CI

Rely on numerical data, rather than “knowledge”

Pattern recognition
(Bezdek, 1994)


Neural, evolutionary and fuzzy systems together

Study of the design of intelligent agents
(Poole, 1998)

Proposed definition: CI is a branch of computer science
studying problems for which there are
no effective
computational algorithms

(Duch, 2007).
Features of CI methods

various methods in CI

share the feature of being subsymbolic

data-driven, where

the structure (knowledge) emerges bottom-up

rather than being imposed from above (pre-wired)

directly draw on environment

Relationship to AI and ML?

Towards foundations of CI

perception, sensory-motor coordination (NNs)

Intermediate level:
fuzzy concepts, associative reasoning

High level:
traditional GOFAI, probabilistic models

Integration of levels necessary in agents

crisis of the richness” (lots of methods available)

Artificial neural networks (MLPs, RBFs, RNNs)

Extended to graphical models

Fuzzy logic – generalization of multi-valued logics

When merged: neuro-fuzzy systems

Evolutionary computing – for complex tasks
(Duch, 2007)
Challenges for CI

no free lunch” theorems
(Wolpert and Macready, 1997)

create flexible systems that can configure themselves

humans are very flexible in finding alternative ways
to solve a given problem

meta-learning research – ability to go beyond
parameter optimization (multiple-expert systems)

Computing and cognition as compression

dimensionality-reducing techniques
Role of semantics

syntax vs. semantics

Including semantics into the system

Symbolic s
– requires an interpreter (symbol grounding problem)

can be added by designer (knowledge base)

stable models, truth values (in a formal system)

Subsymbolic s
– does not (representations are grounded)

How to build it? (interaction with environment)

via search in the model space

requires detailed characterization of the space of
possible models

in ANNs, theoretical results (universal approximation
property) do not guarantee efficient solutions

adaptive systems

homogeneity” of current approaches in terms of

uniform feature space

involved models
Similarity-based framework for meta-learning

chosen metrics is important

Various norms (L1, L2), cosine, scalar product

Using kernels for high-dim. feature spaces

Multiple transformation steps – layered graphical models

Primary goal: discover hierarchical sets of features

hierarchical models developed

MLPs and RBFs – as implementations of sets of
hierarchical fuzzy threshold logic gates
Transformation-based CI theory

Enhancements of feature extraction (from a small array of

separate basic functions and receptive fields for each dimension

scaling the features

creating new features via linear combinations

Extracting information from all features

expansion of the feature space (kernel methods)

increasing data dimension facilitates data separation
Prototype-based rules for data understanding

extraction of logical rules in most machine learning

Problems: many 'natural' categories are not natural

inspiration from human cognition:

human categorization based on memorization of exemplars and

combining feature selection and prototype selection
methods useful

Committees of models (
mixture of experts
Important concepts from computational
learning theory

CLT analyzes the sample complexity an computational
complexity of inductive learning.

trade-off b/w


of hypothesis language and


of learning

bias-variance trade-off

for model estimators

few parameters

igh bias, low variance

too many parameters

high variance, low bias


imposing constraints to avoid overfitting


(probably approximately correct) learning algorithms


(of data) assumption
Open problems for CI

CI that goes beyond probabilistic pattern recognition

Existing methods cannot learn difficult problems

Problems requiring reasoning based on perceptions
should be explored

Methodology of evaluation and development of CI
methods needed

Cognitive robotics may be an ultimate goal