I am not able to get correct output.
MLP not working as expected. Code is same as in video
import numpy as np def softmax(x): e_px = np.exp(x) ans = e_px / np.sum(e_px, axis=1, keepdims=True) return ans def loss(y_ohv, p): return -np.mean(y_ohv * np.log§) def one_hot(y, classes): m = y.shape[0] y_ohv = np.zeros((m, classes)) y_ohv[np.arange(m), y] = 1 return y_ohv class NeuralNetwork: activation_outputs: tuple def init(self, input_size, layers, output_size): np.random.seed(0) model = dict() model[‘w1’] = np.random.randn(input_size, layers[0]) model[‘b1’] = np.random.randn(1, layers[0]) model[‘w2’] = np.random.randn(layers[0], layers[1]) model[‘b2’] = np.random.randn(1, layers[1]) model[‘w3’] = np.random.randn(layers[1], output_size) model[‘b3’] = np.random.randn(1, output_size) self.model = model def forward(self, x): w1, w2, w3 = self.model[‘w1’], self.model[‘w2’], self.model[‘w3’] b1, b2, b3 = self.model[‘b1’], self.model[‘b2’], self.model[‘b3’] z1 = np.dot(x, w1) + b1 a1 = np.tanh(z1) z2 = np.dot(a1, w2) + b2 a2 = np.tanh(z2) z3 = np.dot(a2, w3) + b3 y_ = softmax(z3) self.activation_outputs = (a1, a2, y_) return y_ def backward(self, x, y, learning_rate=0.5): w1, w2, w3 = self.model[‘w1’], self.model[‘w2’], self.model[‘w3’] m = x.shape[0] # Number of Examples a1, a2, y_ = self.activation_outputs delta3 = y_ - y dw3 = np.dot(a2.T, delta3) db3 = np.sum(delta3, axis=0) / float(m) delta2 = (1 - np.square(a2)) * np.dot(delta3, w3.T) dw2 = np.dot(a1.T, delta2) db2 = np.sum(delta2, axis=0) / float(m) delta1 = (1 - np.square(a1)) * np.dot(delta2, w2.T) dw1 = np.dot(x.T, delta1) db1 = np.sum(delta1, axis=0) / float(m) # Applying Gradient Descent self.model[‘w1’] -= learning_rate * dw1 self.model[‘b1’] -= learning_rate * db1 self.model[‘w2’] -= learning_rate * dw2 self.model[‘b2’] -= learning_rate * db2 self.model[‘w3’] -= learning_rate * dw3 self.model[‘b3’] -= learning_rate * db3 def train(x, y, model: NeuralNetwork, learning_rate=0.5, epochs=100, logs=True): training_loss = list() classes = np.unique(y).shape[0] y_ohv = one_hot(y, classes) for ix in range(epochs): y_ = model.forward(x) lo = loss(y_ohv, y_) training_loss.append(lo) model.backward(x, y_ohv, learning_rate) if logs: print(‘Epoch #%d Loss %.4f’ % (ix, lo)) return training_loss
hey @Ritwik-Ranjan-1493378380807281 ,
I just saw your code , it just classes and functions nothing else.
And also to what as incorrect output are you referring too. Kindly let us know.
Whether wrong loss graph or output values are coming very distorted.
Kindly brief more about it.
THank You 
.
This type of graph is coming. I have use make_circles from scikit.databases learning rate was set to 0.001 and epoch was set to 5000. Could you tell me the reason for that.
model = NeuralNetwork(2, [10, 5], 2)
x, y = make_circles(500, noise = 0.05)
a = train(x, y, model, 0.01, 5000)
This was the code for training and then a was plotted against epoch
There are multiple reasons for this.
-> The parameters that you use to generate the data.
-> learning rate
-> number of iterations.
-> proper gradient descent being performed
-> Number of hidden layers
-> particular nodes in hidden layers
There are multiple this.
Can you please provide the actual full code , so that i can debug it .
Like here , you have layers to be 10 and then 5 , and the way you are creating the data.
are you able to debug?
kindly share me full running code , so that i am able to debug.
else if i implement that code , that wont be debugging.
This is the code used to make the MLP and the train function was used for training of data.
https://ide.codingblocks.com/s/371055
Libraries imported
from sklearn.datasets import make_circles
import matplotlib.pyplot as plt
Model Creation and generation of dummy data
model = NeuralNetwork(2, [10, 5], 2)
x, y = make_circles(500, noise = 0.05)
Training the data and storing the loss and generating a graph
a = train(x, y, model, 0.001, 5000)
plt.plot(a)
plt.show()
The Graph achieved
This is all that i have. Please look at the whole picture and tell me what’s wrong with my code.
hey @Ritwik-Ranjan-1493378380807281 ,
your basic code for softmax and training is correct. What you need to do is ,
that according to data we need to modify or tune our model and find the best combination of values that provide us with a good and satisfactory result.
So try this once ,
from sklearn.datasets import make_circles
import matplotlib.pyplot as plt
model = NeuralNetwork(2, [5, 4], 2)
x, y = make_circles(500, noise = 0.2)
a = train(x, y, model, 0.0001, 1000,logs=False)
plt.plot(a)
plt.show()
Creating data , number of nodes , learning _rate , number of iterations all play a very important role.
Although , currently you are working on these only, but in future when you will working on complex deep learning structures , you will get to know about various other parameters too.
I hope this helped you .
1 Like
Thanks for the help!!
No problem buddy.
I guess now your doubt has been resolved , so i would request you to kindly mark it as resolved in courses’s doubt section and do provide your valuable feedback.
Thank You and Happy Learning .