tute4(1)

SC4001 Tutorial 4 Deep neural networks Figure 1 1. The two-layer feedforward perceptron network shown in figure 1 has weights and biases initialized as indicated and receives 2-dimensional inputs (࠵? ! , ࠵? " ) . The network is to respond with ࠵? ! = ’ 0 1 * and ࠵? " = ’ 1 0 * for input patterns ࠵? ! = ’ 1.0 3.0 * and ࠵? " = ’ −2.0 −2.0 * , respectively. Analyse a single feedforward and feedback step for gradient decent learning of the two patterns by doing the following: (a) Find the weight matrix ࠵? to the hidden-layer and weight matrix ࠵? to the output-layer, and the corresponding biases. (b) Calculate the synaptic input ࠵? and output ࠵? of the hidden-layer, and the synaptic input ࠵? and output ࠵? = (࠵? ! , ࠵? " ) of the output layer. (c) Find the mean square error cost ࠵? between the outputs and targets. (d) Calculate the gradients ࠵? # ࠵? and ࠵? $ ࠵? at the output-layer and the hidden-layer, respectively. (e) Compute the new weights and biases. (f) Write a program to continue iterations until convergence and find the final weights and biases. Assume a learning rate of 0.05. Repeat above (a) – (f) for stochastic gradient decent learning. ࠵? ! +1 ࠵? " +1 1 2 -2 0 3 -1 1 1 0 -2 -2 3 ࠵? ! ࠵? "