Assignment 2: Neural Network analysis -- German Credit Data Due: Monday October 31

sciencediscussionΤεχνίτη Νοημοσύνη και Ρομποτική

20 Οκτ 2013 (πριν από 3 χρόνια και 9 μήνες)

181 εμφανίσεις

Assignment 2
:
Neural Network

analysis
--

German Credit Data

Due: Monday October 31
st




D
evelop a neural network classification mo
del for the German Credit data (used in
Assignment 1 for decision tree models)



We should extract 3
0% of the total observat
ions (1000 total observations) for the
test

set
.


In your experimentation, examine the following features




N
etwork configuration: number of hidden layers (1
-
2), number of neurons in each
hidden layer.
Try the default setting (with the Quick method), and th
en with two other
configurations.

(Note: greater number of hidden layer neurons can lead to over
-
fitting! What
guidelines do you use
, and what are your conclusions?
)


Also try the Dynamic method and the Multiple method (do you find this feature
useful?)




L
earning rate and momentum values (systematically experimentation with some
values of these).




S
topping conditions (You can use performance on the validation data to determine
when overfit sets in
-

is this feature helpful? Note that final performance is
determined from that on the test data set.)




Different initial random weights


does this make a difference?




Try training the NN using balanced data (equal proportions of the two classes).


How sensitive is the neural network to training data size


try u
sing 50% of data for
training, and 50% for testing, also 80% for training and 20% for testing. (Note how the
NN node in Clementine can be set to use a set percentage of the training data for
validation through the “Prevent Overtraining” option)


What is t
he best performance model that you obtain?

How is ‘best’ judged? Assess
performance on accuracy as well as on lifts.


How does
your NN model performance

compare with
that of
the decision tree model
from Assignment 1?

Considering sensitivities of variable
s in the NN model, is there any
correspondence with the variables in the decision tree?