# The backpropagation algorithm

T

Backpropagation is one of the several ways in which an artificial neural network (ANN) can be trained. It is a supervised training scheme, which means, it learns from labeled training data. In simple terms, BackProp is like “learning from mistakes. “The supervisor corrects the ANN whenever it makes mistakes.”

Initially, all the edge weights are randomly assigned. For every input in the training dataset, the ANN is activated, and its output is observed. This output is compared with the desired output that we already know, and the error is “propagated” back to the previous layer. This error is noted, and the weights are “adjusted” accordingly. This process is repeated until the output error is below a predetermined threshold.

What’s special about it, is the way the computations are carried out: each layer sees the error propagated backward into it as a ‘black box’ (a pre-computed value), which makes the computation in each layer a ‘local’ one, and therefore simplifies the whole process. This also gives way to implementing the algorithm in a computationally efficient way, which conceptually resembles the idea behind dynamic programming.

The following section describes the Backpropagation algorithm.

```Assign all network inputs and output
Initialize all weights with small random numbers, typically between -1 and 1

repeat

for every pattern in the training set

Present the pattern to the network

//        Propagated the input forward through the network:
for each layer in the network
for every node in the layer
1. Calculate the weight sum of the inputs to the node
2. Add the threshold to the sum
3. Calculate the activation for the node
end
end

//        Propagate the errors backward through the network
for every node in the output layer
calculate the error signal
end

for all hidden layers
for every node in the layer
1. Calculate the node's signal error
2. Update each node's weight in the network
end
end

//        Calculate Global Error
Calculate the Error Function

end

while ((maximum  number of iterations < than specified) AND
(Error Function is > than specified))```

### References

http://www.cse.unsw.edu.au/~cs9417ml/MLP2/

Backpropagation for dummies

https://www.quora.com/How-do-you-explain-back-propagation-algorithm-to-a-beginner-in-neural-network