ICS 586: Neural Networks

ocelotgiantAI and Robotics

Nov 7, 2013 (3 years and 5 months ago)

78 views

ICS 586: Neural Networks

Dr. Lahouari Ghouti

Information & Computer Science Department

ICS 586: Neural Networks

First Semester 2008/2009 (081)



Instructor:

Dr. Lahouari Ghouti

Information and Computer Science Department

Email: lahouari@kfupm.edu.sa

Office: Building 22


Room 128

Tel: 1922

Grading Policy

Task

Weight

Four Quizzes

10%

Homeworks

10%

Research Paper (Lecture) Presentation

10%

One Major Exam

15%

Final Exam

25%

Research Project

[Proposal
5%
-

Final Report + Prototype
25%

-

Class Presentation
10%
]

40%

Tentative Schedule



Introductory Meeting [Introduction to Neural Networks]



Single Layer Perceptron



Multilayer Perceptron



ADALINE



The LMS Algorithm



Backpropagation Learning



Overfitting, Cross
-
Validation, and Early Stopping



Simple Recurrent Networks



Pattern Classification (Guest Speaker?)



Radial Basis Functions



Support Vector Machines



Competitive Learning and Kohonen Nets



Hebbian Learning



Principal Components Analysis (PCA)



Adaptive Principal Components Extraction (A Student)



Non
-
Negative Matrix factorization (A Student)



Hopfield Networks and Boltzmann Machines



Bayesian Networks (A Student)



Hidden Markov Models (A Student)


Extreme Learning Machines (A Student)

Programming Environment

We will be using MATLAB in this course.

If you know, that’s fine

If you do not, you will need to learn it.

Getting Started With ANNs

Foundations of Neural Computation:


Understand the operation of single neurons or small neural circuits.


Detailed biophysical models of nerve cells (receptors, ion channels,
membrane voltage), and collections of cells.

Varieties of “Neural Networks” Research

1
-

Neuronal Modeling


2
-

Computational Neuroscience


3
-

Connectionist / Parallel Distributed Processing (PDP) Models


4
-

Artificial Neural Networks (ANNs)

Connectionist (PDP) Modeling

Model human cognition in a brain
-
like way:




Massively parallel constraint satisfaction.




Distributed activity patterns instead of symbols.




Models are fairly abstract.

ANN Landscape

Artificial Neural Networks

Models:

Simple, abstract, .neuron
-
like. computing elements; local computation.











Applications:


Pattern recognition, adaptive control, time series prediction.

This is where the
money

gets made!

Reference: Pomerleau 1993: ALVINN

Artificial Neural Networks: The Beginnings

W. S. McCulloch and W. Pitts (1943) Logical calculus of the ideas
immanent in nervous activity. Philosophy of Science 10(1), 18
-
24.

Warren McCulloch

Walter Pitts

Revolutionary Idea:

think of neural tissue as circuitry performing
mathematical computations!

The McCulloch
-
Pitts Neuron

Linear weighted sum

of inputs:

Learning rule:


Nonlinear
, possibly
stochastic

transfer function
:

Transfer function
g(x)



i
i
i
x
w
netact


netact
g
y




i
w
What to do now?



Check WebCT for course syllabus + soft copy of textbook




Start learning MATLAB if you do not know it!




Select a topic you want to present in the class from the tentative
schedule (first
-
come first
-
serve basis!)




Select your time slot to discuss with me your project. The earlier you
start, the better off you will be.