Description
Question 1. [100 points]
Implement a two-layer perceptron with the backpropagation algorithm to solve the parity problem.
You must implement the forward and backpropagation paths entirely on your own,
including an implementation of a perceptron and the activation function. The desired
output for the parity problem is 1 if an input pattern (which contains 4-binary bits) contains an odd
number of 1’s, and 0 otherwise. Follow the algorithm introduced in class and consult the textbook.
Use a network with 4 binary input elements, 4 hidden units for the first layer, and one output unit
for the second layer. The learning procedure is stopped when an absolute error (difference) of 0.05
is reached for every input pattern. Other implementation details are:
• Initialize all weights and biases to random numbers between -1 and 1.
• Use a logistic sigmoid with a = 1 as the activation function for all units.
After your basic programming is done, do the following:
1. Vary the value of η from 0.05 to 0.5 using an increment of 0.05, and report the number
of epochs needed for each choice of η. Discuss how the value of η influences convergence.
Generate a few learning curve plots to support your discussion.
2. Include a momentum term in the weight update with α = 0.9 and report its effect on the
speed of training for each value of η.
2/2
