Back Propagation Lab
The Lab
Now that we all understand the basic functioning of the backpropagation
network. Let's start the simulation!
Double click on the Backprop.exe file that you copied to your hard drive.
Once the program has loaded click on the file menu and select new network.
The program will ask you to specify the number of neurodes for the input,
middle, and output layer. Using the number keys at the top of your main
keyboard and the enter key select 5 by 7 for the input layer, 2 by 5 for
the middle layer and 1 by 8 for the output layer. Also in the file menu,
select set input and hit the return key to select the file "ACEQSTV.DAT".
In the run menu, select mode and then select Continuous. Turn training
on in the run menu. To turn training on, make sure that the training box
says "turn training off") and check to make sure the noise is off (if not
turn it off in the set menu). Check the status box in the lower left hand
corner to make sure that all the parameters are correct and that the learning
constant is set at .5.
The system should look like this:
You are going to train the system, using the figures and answers in the
"ACEQSTV.DAT" file, to output the ASCII number for the letter that is input
in the layer. To see that your system is not trained set the training off,
set the mode for one pattern, and select start. The input should have a
pattern of a letter, the answer key should represent the desired output and
the middle and output layers should have some unrecognizable pattern. The
diagram below shows a typical input pattern and answer.
Now, to begin your training, turn the training on, set the mode for Continuous,
and select start. As the training begins, two new diagrams appear in the
lower right hand corner of the screen. These are strip-chart recorders.
The top-most recorder keeps track (qualitatively) of the total error per
pass through the complete training set. The Lower-most recorder keeps track
of the error for each pattern (qualitatively). There are also numbers for
the worst error for a pattern on the last pass in the left-hand box. You
might want to wait until the system finishes and then initialize the network
and start over to answer the question. You can even set the mode for one
pass and manually run the system through a few passes
1.) After a few passes through the training file has the total error decreased
substantially?.
Has the error on individual patterns decreased substantially?
Does the error decrease evenly across all patterns during training?
2.) Allow the network to train to completion. (Set mode for continuous)
The simulator stops once the worst error for all of the elements of the
pattern drops below .1. The status box keeps track of the number of passes
through the training set. How many passes did the network have to be trained
for before it learned them all?
3.) Once the training has stopped set the mode for one pattern. The simulator
will then process one pattern at a time. Turn the training off to prevent
further weight change. Next select start on the run menu repeatedly until
you have processed the entire data set one pattern at a time. Does the network
know all the patterns? It will be helpful here if you state a criteria for
correct response, like "very little or no noise on nodes that are blank in
the desired output pattern and full or nearly full activation of activated
nodes for pattern.
4.) In the Set menu, turn the noise on, set the noise level to 20%. This
adds a random number in the range of ±.2 to each input pattern element.
Are there any patterns to which it cannot correctly respond? Which ones?
Start the simulator repeatedly until you have reviewed the training set.
Can the network respond correctly to the noisy input?
5.) Run through the training patterns a couple of times to see how consistently
the network handles them. Then increase the noise to each of the following
values and fill out the chart below for a single pass through the training
set.
Noise level | 20% | 40% | 60% | 100% |
Total Missed Patterns |
6.) Summarize how well you think the network learned these patterns.
Is the network robust in its processing of each image?
7.) Turn the noise off, initialize the network in the run menu, turn the
training on, set the mode for Continuous. In the Set menu, choose Set Parameters
and enter .1 for the learning constant . Repeating steps 2-6, fill out the
form below for each value of .
Training constant value | Passes to Train | Noise level 20% #Missed | Noise level 40% #Missed | Noise level 60% #Missed | Noise level 100% #Missed |
Training constant value .1 # missed | |||||
Training constant value .5 # missed | |||||
Training constant value .7 # missed | |||||
Training constant value .9 # missed |
8.) Did the network ever fail to learn the patterns? If so were you able
to reinitialize the network and retrain it correctly?
9. Initialize the network and set the learning constant to .5. Train
the network and note the number of passes though the training set needed
to learn. Is it the same as before? Repeat this step several times. Was
the number the same? Why or why not?