Case Study - GoogleNet, Inception Module

#1

I didn’t understand vanishing gradient problem and how GoogLeNet resolves this ,plz explain

#2

Hey @Bhawna,

Vanishing Gradient Problem occurs when we try to train a Neural Network model using Gradient-based optimization techniques.

Generally, adding more hidden layers tends to make the network able to learn more complex arbitrary functions, and thus do a better job in predicting future outcomes. This is where Deep Learning is making a big difference due to the thousands and millions of hidden layers it has, we can now make sense of highly complicated data such as images, speeches, videos, etc and do Speech Recognition and Image Classification, Image Captioning, etc.

Now coming to the point- What is the Vanishing Gradient Problem?

Now when we do Back-propagation i.e moving backward in the Network and calculating gradients of loss(Error) with respect to the weights, the gradients tends to get smaller and smaller as we keep on moving backward in the Network. This means that the neurons in the Earlier layers learn very slowly as compared to the neurons in the later layers in the Hierarchy. The Earlier layers in the network are slowest to train.

The problem of Vanishing Gradient could be resolved by using the ReLU activation function.

ReLU

The ReLU or Rectified Linear Unit is represented as ReLU(z) = max(0, z). It basically thresholds the inputs at zero, i.e. all negative values in the input to the ReLU neuron are set to zero.

I hope this clears all your doubts.

If you are satisfied with the answer you can mark doubt as resolved and can rate your experience.