# WP.SEA.4.1 Tutorial on Statistical Learning Theory 2006

AI and Robotics

Oct 16, 2013 (4 years and 9 months ago)

96 views

WP.
SE
A.
4
.1

Tutorial
on
Statistical Learning Theory

2006

Lecturer
:

Dr.
A. Chervonenkis
, ICS, RAS, partner 41
,

Keywords :

S
tatistical learning theory,

support vector machine, pattern recognition,

intrusion detection

Venue :

WP.
JRA.5.1.meeting
,
07.11.2006, CWI, Amsterdam

Theme of the course

O
verview o
f

statistical learning theory

Report:

D.JRA.5.1.3

Further m
aterial: available
from

on request

Program of the
tutorial

In
Statistical Learning Th
eory

machine learning is considered as a search of a decision rule or
a dependence between input and output data minimizing average risk of errors based on a
training set. The resulting rule or dependence is searched within some class of possible
variants.

The main statistical problem here is that if the class is too large, information
contained in the learning set may be not sufficient to choose the proper result; on the contrary
if the class is too small, it is possible, that there is no proper decision w
ithin the class. Various
methods of regularization are used to find the optimal compromise.

In
pattern recognition

linear decision rules are often useful. A method for searching linear
decision rule with regularization called “Generalized Portrait” was pro
posed by V.N. Vapnik
and A.Chervonenkis

in the sixties. The distance between the separating hyperplane and the
sets presenting classes of patterns versus the number of errors on the training set was used as a
regularization factor. It appeared that only so
me subset of the training set (called the set of
support vectors) totally determines the result. All the technique in this method could be
expressed in terms of an inner (scalar) product.

Later V.N. Vapnik proposed to use instead of the usual inner product

arbitrary positively
defined kernels (all other technique was inherited from the “
Generalized Portrait
”). It allowed
to include in the area of search also non
-
linear decision rules. This approach is known as
Support Vector Machine
. Many kinds of kernels w
ere proposed and used in various fields of
application.

In the tutorial the review of the Support Vector Machine history and technique, fields of its
application and different kinds of kernels is given. Application for Non Authorized Intrusion
Detection is

considered in more details.