Not able to understand get_grads

#1

i am not able to understand get_grads function at all

#2

what is grad_w and gra??d_b

#4

Hello @parth_99,

The get_grads function you mentioned looks like this, lets understand it line by line.

1  def hypothesis(x,theta):
2   return sigmoid(np.dot(x,theta))
3
4  def sigmoid(z):
5    return 1/(1+np.exp(-z))
6
7  def get_grad(Y,X,theta):
8    
9      grad = np.zeros(theta.shape)
10     m = X.shape[0]
11    for i in range(m):
12         y_hat = hypothesis(X[i],theta)
13         x = X[i].reshape((-1,1))
14         grad += (Y[i] - y_hat)*x
15       
16        
17     return grad/m #Average Gradient

Here the function hypothesis returns P, where P is the probability of X being 1 or it being to class 1.
Now the get_grads,

This function returns the gradient

in line 9:

We have initialized gradient of theta or you can say the parameters of the decision boundary (here in our case a line) to zero.

in line 11:

We are iterating through our whole dataset.
y_hat is the prediction of our model of X using theta.

in line 13:

We have reshaped X [ i ] into a column vector so as to multiply it with Y [ i ] such that there is no need for an explicit sum function.
Note: Try doing the same without line 13, you can see the difference.

in line 14:

We have calculated the gradient of the log loss function. If you calculate the gradient of the log likelihood/log loss function with respect to theta, you will get the same result, i.e (y_pred - y_true) * X

in line 17:

We here have averaged the gradients and returned the values. (Try doing this without this averaging, you will see some interesting results.)

I hope with this you got an understanding into what is get_grads doing. Any query with any of the things mentioned, feel free to continue here.

Happy Learning :slight_smile:
Thanks