back propagation algorithm example, Summaries of Pattern Classification and Recognition

Download back propagation algorithm example and more Summaries Pattern Classification and Recognition in PDF only on Docsity!

3 Back Propagation (BP) Algorithm One of the most popular NN algorithms is back propagation algorithm. Rojas [2005] claimed that BP algorithm could be broken down to four main steps. After choosing the weights of the network randomly, the back propagation algorithm is used to compute the necessary corrections. The algorithm can be decomposed in the following four steps: i) Feed-forward computation ii) Back propagation to the output layer iii) Back propagation to the hidden layer iv) Weight updates The algorithm is stopped when the value of the error function has become sufficiently small. This is very rough and basic formula for BP algorithm. There are some variation proposed by other scientist but Rojas definition seem to be quite accurate and easy to follow. The last step, weight updates is happening through out the algorithm. BP algorithm will be explained using exercise example from figure 4. 3.1 Worked example NN on figure 4 has two nodes (NO0,0 and NO,1) in input layer, two nodes in hidden layer (N1,0 and N1,1) and one node in output layer (N2,0). Input layer nodes are connected to hidden layer nodes with weights. (W0,1-W0,4). Hidden layer nodes are connected with output layer node with weights (W1,0 and W1,1). The values that were given to weights are taken randomly and will be changed during BP iterations. Table with input node values and desired output with learning rate and momentum are also given in figure 5. There is also sigmoid function formula f(«) = 1.0/(1.0 + exp(—a)). Shown are calculations for this simple network (only calculation for example set 1 is going to be shown (input values of 1 and 1 with output value 1)). In NN training, all example sets are calculated but logic behind calculation is the same. 3.1.1 Feed-forward computation Feed forward computation or forward pass is two step process. First part is getting the values of the hidden layer nodes and second part is using those values from hidden layer to compute value or values of output layer. Input values of nodes NO,0 and NO,1 are pushed up to the network towards nodes in hidden layer ( N1,0 and N1,1). They are multiplied with weights of connecting nodes and values of hidden layer nodes are