as2/CS 6665 Homework 2x - Chad Maughan

madbrainedmudlickAI and Robotics

Oct 20, 2013 (3 years and 9 months ago)

71 views

CS 6665

Homework 2



100

points
-

turn in through eagle
, preferably as a .pdf

. Due midnight,
2/10
/2011.


To turn in a homework through eagle you must first add yourself as a user,
and place yourself in
the CS6665

class (
eagle.cs.usu.edu
).
You should do
this before the due date. If you have
troubles, it takes at least 48 hours (not counting the weekend) to correct problems, so plan ahead.


With homework 1, you examined the use of principle components to reduce the number of
attributes in your input vector
. For this homework, you will use the Matlab neural network tool
nftool

to build a neural network for recognition.
Remember, each record of the data file of
Homework 1 included a category M for malignant and B for benign.
In all of what follows, the
assump
tion is that your initial data file is the same as for the first homework


H1Data.xls, and it
is assumed that you have already found the principle components as:



[COEFF,SCORE,latent]=princomp(X);


X is H1Data with only

the attribute columns included, i.
e. columns 1(patient id) and 2 (the
classification M or B

NOT included
)


1.

Your first task will be to

read in the data file H1Data.xls.


2.

Next, you need to create two tables, X and Y. X should be a table 569X30 taken from
H1Data.xls. In this case, X is simply

the last 30 columns of H1Data, and thus it is the
attribute columns.

Y will be
a mat
rix which corresponds only to
column 2 of H1Data.xls.
This is the column whose values are all M or B. You will need to convert these values to
1 for M and 0 for B
It is po
ssibly easier to edit this column in H1Data before reading
H1Data into Matlab. This is because when you read in H1Data.xls, this column is read

as
NaN. The table for this data

will be

Y

(569X1)
.


3.

You next need to perform the following runs of nftool

a.

X

as
input with original (unchanged) attribute values
, and Y as the target or
output values. Perform three

runs using

different numbers

of neurons in the
hidden layer. Use 55% of the data for training, 15% for validation, and 30% for
testing.

For each of the ne
uron configurations, perform 3 runs. Thus you will have
9 runs for this part. Save the network output that has the best MSE.


b.

X

as input with original attribute values changed to 0 mean a 1 STD
, and Y as the
target.
Perform three runs using the same number
s of neurons in the hidden layer
as in part a. Use 55% of the data for training, 15% for validation, and 30% for
testing.

For each of the neuron configurations, perform 3 runs. Thus you will have
9 runs for this part. Save the network output that has the b
est MSE.


c.

X

converted to a single attribute using the highe
st variance principle component,
and Y as the target.
Perform three runs using the same numbers of neurons in the
hidden layer as in part a. Use 55% of the data for training, 15% for validation, an
d
30% for testing.

For each of the neuron configurations, perform 3 runs. Thus you
will have 9 runs for this part. Save the network output that has the best MSE.


d.

X

converted to the N princip
le components as chosen using your

scree plot of
homework 1, and Y

as the target. Perform three runs using the same numbers of
neurons in the hidden layer as in part a. Use 55% of the data for training, 15% for
validation, and 30% for testing.

For each of the neuron configurations, perform

3
runs. Thus you will have 9 runs for this part. Save the network output that has the
best MSE.



F
or the four parts (a,b,c,d) there should be at least 36 total runs of nftool.


The output of the neural network is a continuous value. Often, when the targe
ts are discrete
values such as 0/1, a round or truncation is done on the output to get an actual predictor. Nftool
does not do such operations when continuous transfer functions are used. For this step, you are to
take the output of each of your “best” ne
t
works

in terms of MSE

found in a
-
d above. U
se the
following two distinct rules to do the transform of the output:



Rule 1: if output>= 0.5 then output =1




e
lse





Output = 0



Rule 2:


if output >=0.6 then output =1




e
lse





If output
<=0.4 then ou
tput =0





e
lse






output=
-
1

(undecided)



Having done all of this, you should then turn in the following table, in .pdf
.
The columns are
defined as follows:

(assume the hidden layer neuron choices are 3, 7, and 20.


Run #

Run 11a denotes the first run

for question #a using 3 neurons

Run 12a denotes the second run for question #a using 3 neurons

Run 13a denotes the third run for question #a using 3 neurons.

Run 21a denotes the first run for question #a using 7 neurons

Run 22a denotes the second run for
question #a using 7 neurons

Run 23a denotes the third run for question #a using 7 neurons.

.

.

Run 11b denotes the first run for question #b using 3 neurons

.

.

Run 33d denotes the last run for question

#d using 20 neurons.



# Hidden Layer Neurons:

For e
ach of the three networks choose a set number of neurons for the hidden layer. For
example, if you choose 3, 7, and 20, then the first three rows o
f the table, i.e. Run # 11a,
12a, and 13a

would have the value 3 for the # Hidden Layer Neurons. The next thr
ee
rows w
ould be for question a using 7 neurons in the hidden layer, etc.


MSE Test:

This is the mean squared error rate on the test data for the given network.


Rule 1 # Correct:

For the first three rows, only one will be filled in with a value and it wil
l be the #correct
answers on the test data (not the training or validating data), etc.


Rule 1 #Incorrect:

For these three rows, only one will be filled in and it will be the #incorrect answers on the
test data (not the training or validating data), etc.


Rule #2:

These columns will be filled in the same as Rule #1, except there is a column for the
number incorrect. Remember, in Rule#2, if the network output is between 0.6 and 0.4,
then the network is undecided as to whether to output a 1 or 0.






Rule 1

Rule 2


Run #

# Hidden

Layer Neurons

MSE

Test

#
Correct

#
Incorrect


#

Correct


#

Incorrect


#

Undecided

Run 11a








Run 12a








Run 13a








Run 21a








.








.








Run 33a








Run 11b








.








.








Run 33d