Neural Networks Applications

munchsistersAI and Robotics

Oct 17, 2013 (3 years and 5 months ago)

57 views


1

Neural Networks Applications

Final

20
11

Fall

Key

2010/6
/
16


1.

(24
%)
Terminology

a.

Radial basis function

b.

Self
-
organization feature map

c.

T
wo
-
third rule

d.

V
-
C dimension

e.

Support vectors

f.

Empirical
Risk


2.

(
1
0
%)
For the figure shown as follows,



Please answer the following questions:


a.

(4%)
What is the meanings of X and Y axes?


2

b.

(3%)
What class that the new pattern(1.5) should be assigned ?

c.

(3%)
What is the meaning of x
d
?

Ans:

a.

X axis indicates the
value of a new pattern

b.

It should be assigned to C1

c.

x
d

is the discriminant function

that
point x lies in R
1

(decide C
1
) if
p(x|C
1
)P(C
1
) > p(x|C
2
)P(C
2
)

and
point x lies in R
2

(decide C
2
) if p(x|C
2
)P(C
2
)
> p(x|C
1
)P(C
1
)



3.

(5
%)
For the following two figures, wh
y do the two Bayesian decision boundary
functions differ?


3


Ans:

In the first figure, the covariance of these two classes are equal. On the contrary,
the covariance of the two classes are unequal.


4.

(
16
%)
Consider the two class data: Class 1={(1.5, 1),
(1.5, 1.5), (2, 1), (2, 1.5)},
Class 2={(
-
1, 0), (0,
-
1), (0, 1), (1, 0
)
}.

a.

Estimate the means and covariance matrices of both distributions.

b.


Find the equation of the Bayesian decision boundary assuming P(C
1
)=0.4,
P(C
2
)=0.6.

Ans:

a. mean







covariance







4









b.

Discriminant functions:


F
or class 1


For class 2


The discriminant function:


P(C
1
)=0.4, P(C
2
)=0.6




5.

(10%)
How can one view RBFN designs as a special case of support vctor
learning?


Ans:


The selection of a proper kernel is critical to the problem of maximizing the
performance

of an SVM. When we have kernel as




(



)



(










then this yield Gaussian radial basis function classifiers. For the case of
Gaussian radial

basis function classifiers the support vector machine is indeed a
radial basis functionnetwork where the centers is the number of support vectors,


5

and the weights
(
λ
i
)
and

bias are chosen automatically using SVM learning
procedure.


6.

(10%) The margin plays a key role in the design of support vector machines.
Please identify the important properties of the margin in solving
patter
-
classification probl
ems.


Ans:



There are two cases to be considered:


Case I
:
Linearly Separable Patterns


In this case, optimality of the
margin
of separation between the two classes of

data points is

unique
. Moreover, the decision boundary separating the two

classes consists of a
hyperplane
.

Accordingly, design of the support vector

machine boils down to Rosenblatt’s perceptron. In other

words, there is no need

for a hidden layer or feature space.


Case II
:
Nonseparable Patterns


In this second case, comput
ing the optimal decision boundary is
no
longer

unique.

Correspondingly, we speak of a
soft margin
. The exact form of the

optimal decision boundary

depends on two factors:


• the user
-
specified parameter
C
, and


• the actual data points themselves.

Hence,

the soft margin varies from one design to another.

Moreover, the optimal
decision boundary is nonlinear, which means that the support

vector machine
requires the inclusion of a hidden layer to separate the two classes, one from the

other.


7.

(10%)
The next
figure shows a data set that consists of nonl
i
nearly separable
positive and negative examples.
Specifically
, the decision boundary separating
the positive and negative examples is an ellipse modeled by

x
1
2
a
2
+
x
2
2
b
2
=1




Negitive






Positive



6





Please find
and plot
the transformation that maps the positive and negative
examples become linearly separable in the feature space.


Ans:

Let

y
1


1


y







So

y
1
a
2
+
y
2
b
2
=1

That is,

y


b

(
1

y
1
a

)






b
2


Negative



Positive









a
2



8.

(
15
%)Perform three steps of the discrete time learning algorithm of the ART1
network for input neuron n=9 and output neuron m=3 assuming that the training
vectors are:


x
1
=[1 0 0 0 1 0 0 0 1]


x
2
=[1 1 0
0 1 0 0 1 1]


x
3
=[1 0 1 0 1 0 1 0 1]


Please select an appropriate vigilance level ρ so that all three patterns are clustered
separately when the sequence x
1
, x
2
, x
3
, is presented. Compute the final weights
W
, and
V

of the trained network.