Backpropagation is one of the several ways in which an artificial neural network (ANN) can be trained. It is a supervised training scheme, which means, it learns from labeled training data. In simple terms, BackProp is like “learning from mistakes. “The supervisor corrects the ANN whenever it makes mistakes.”

Initially, all the edge weights are randomly assigned. For every input in the training dataset, the ANN is activated, and its output is observed. This output is compared with the desired output that we already know, and the error is “propagated” back to the previous layer. This error is noted, and the weights are “adjusted” accordingly. This process is repeated until the output error is below a predetermined threshold.

What’s special about it, is the way the computations are carried out: each layer sees the error propagated backward into it as a ‘black box’ (a pre-computed value), which makes the computation in each layer a ‘local’ one, and therefore simplifies the whole process. This also gives way to implementing the algorithm in a computationally efficient way, which conceptually resembles the idea behind dynamic programming.

The following section describes the Backpropagation algorithm.


Backpropagation for dummies


Write A Comment