1
Neural Networks Applications
Final
20
11
Fall
Key
2010/6
/
16
1.
(24
%)
Terminology
a.
Radial basis function
b.
Self

organization feature map
c.
T
wo

third rule
d.
V

C dimension
e.
Support vectors
f.
Empirical
Risk
2.
(
1
0
%)
For the figure shown as follows,
Please answer the following questions:
a.
(4%)
What is the meanings of X and Y axes?
2
b.
(3%)
What class that the new pattern(1.5) should be assigned ?
c.
(3%)
What is the meaning of x
d
?
Ans:
a.
X axis indicates the
value of a new pattern
b.
It should be assigned to C1
c.
x
d
is the discriminant function
that
point x lies in R
1
(decide C
1
) if
p(xC
1
)P(C
1
) > p(xC
2
)P(C
2
)
and
point x lies in R
2
(decide C
2
) if p(xC
2
)P(C
2
)
> p(xC
1
)P(C
1
)
3.
(5
%)
For the following two figures, wh
y do the two Bayesian decision boundary
functions differ?
3
Ans:
In the first figure, the covariance of these two classes are equal. On the contrary,
the covariance of the two classes are unequal.
4.
(
16
%)
Consider the two class data: Class 1={(1.5, 1),
(1.5, 1.5), (2, 1), (2, 1.5)},
Class 2={(

1, 0), (0,

1), (0, 1), (1, 0
)
}.
a.
Estimate the means and covariance matrices of both distributions.
b.
Find the equation of the Bayesian decision boundary assuming P(C
1
)=0.4,
P(C
2
)=0.6.
Ans:
a. mean
covariance
4
b.
Discriminant functions:
F
or class 1
For class 2
The discriminant function:
P(C
1
)=0.4, P(C
2
)=0.6
5.
(10%)
How can one view RBFN designs as a special case of support vctor
learning?
Ans:
The selection of a proper kernel is critical to the problem of maximizing the
performance
of an SVM. When we have kernel as
(
)
(
‖
‖
then this yield Gaussian radial basis function classifiers. For the case of
Gaussian radial
basis function classifiers the support vector machine is indeed a
radial basis functionnetwork where the centers is the number of support vectors,
5
and the weights
(
λ
i
)
and
bias are chosen automatically using SVM learning
procedure.
6.
(10%) The margin plays a key role in the design of support vector machines.
Please identify the important properties of the margin in solving
patter

classification probl
ems.
Ans:
There are two cases to be considered:
Case I
:
Linearly Separable Patterns
In this case, optimality of the
margin
of separation between the two classes of
data points is
unique
. Moreover, the decision boundary separating the two
classes consists of a
hyperplane
.
Accordingly, design of the support vector
machine boils down to Rosenblatt’s perceptron. In other
words, there is no need
for a hidden layer or feature space.
Case II
:
Nonseparable Patterns
In this second case, comput
ing the optimal decision boundary is
no
longer
unique.
Correspondingly, we speak of a
soft margin
. The exact form of the
optimal decision boundary
depends on two factors:
• the user

specified parameter
C
, and
• the actual data points themselves.
Hence,
the soft margin varies from one design to another.
Moreover, the optimal
decision boundary is nonlinear, which means that the support
vector machine
requires the inclusion of a hidden layer to separate the two classes, one from the
other.
7.
(10%)
The next
figure shows a data set that consists of nonl
i
nearly separable
positive and negative examples.
Specifically
, the decision boundary separating
the positive and negative examples is an ellipse modeled by
x
1
2
a
2
+
x
2
2
b
2
=1
Negitive
Positive
6
Please find
and plot
the transformation that maps the positive and negative
examples become linearly separable in the feature space.
Ans:
Let
y
1
1
y
So
y
1
a
2
+
y
2
b
2
=1
That is,
y
b
(
1
y
1
a
)
b
2
Negative
Positive
a
2
8.
(
15
%)Perform three steps of the discrete time learning algorithm of the ART1
network for input neuron n=9 and output neuron m=3 assuming that the training
vectors are:
x
1
=[1 0 0 0 1 0 0 0 1]
x
2
=[1 1 0
0 1 0 0 1 1]
x
3
=[1 0 1 0 1 0 1 0 1]
Please select an appropriate vigilance level ρ so that all three patterns are clustered
separately when the sequence x
1
, x
2
, x
3
, is presented. Compute the final weights
W
, and
V
of the trained network.
Comments 0
Log in to post a comment