Backpropagation

#1

Can u plz explain what exactly is Backpropagation especially "backpropagate loss ",I have studied but still confused .

#2

I would like to talk about linear regression first and taking that as reference i would move to backpropogation.
The steps followed in linear regression (After all preprocessing) are:

  1. Initialize random values of all parameters , namely weight matrix and bias.
  2. Calculate error
  3. Calculate gradients of individual thetas
  4. Apply gradient descend and update parameters, so that your line becomes more close to optimal lines.
  5. Loop on until validation score keeps on decreasing.
    Hope this was already cleared to you.

Now moving to Convolution Neural Networks or simply neural networks, the steps are nearly same as follows:

  1. Initialize random values of all parameters, namely kernels including bias terms.
  2. Calculate error
  3. Calculate gradients of individual parameters, and here we use technique named backpropagation.
  4. Apply gradient descend and update parameters, so that your line becomes more close to optimal lines.
  5. Loop on until validation score keeps on decreasing.

Now since calculating gradients in case of neural network, we use a technique named back propagation, where in we calculate gradient of last layers first and than slowly moves backward for previous layers, by applying chain rule again and again. And after calculating these gradients you can now easily update them. That is the reason it is not as backpropagation since you are propagating from backward.

Hope this helped :blush:

#3

We are doing same thing just we are calculating optimal value of theta of initial layers using last layer?

#4

Yes we are trying to calculate optimal values of all layers including last layers by calculating gradients from loss function, and updating them from backward.

#5

got it thank u…

#6

I hope I’ve cleared your doubt. I ask you to please rate your experience here
Your feedback is very important. It helps us improve our platform and hence provide you
the learning experience you deserve.

On the off chance, you still have some questions or not find the answers satisfactory, you may reopen
the doubt.