Validation accuracy remains constant in LSTM

#1

  1. My training loss, training accuracy and validation loss are changing while training an LSTM for the ‘Emoji Prediction’ project. But my validation accuracy remains constant.

    1601×534 66.6 KB

  2. Even my training accuracy is not very high. Max training accuracy achieved is 0.40 whereas in the video, Prateek Bhaiya is getting 0.96 training accuracy.

This is the model I’m using.

model = Sequential()

model.add(LSTM(64, input_shape=(10,50)))

model.add(Dropout(0.5))

model.add(Dense(5))

model.add(Activation(‘softmax’))

model.compile(loss=‘categorical_crossentropy’, optimizer=‘adam’, metrics=[‘accuracy’])

#2

hey @Jalaanchal-Tewari-1816244721737412 ,
I got what actual problem you are facing . But to be more sure about it , i need to debug your code once.
So can you just upload your code on github or drive and share its me link. It would be really helpful.

Thank You :slightly_smiling_face:.

#3

https://drive.google.com/file/d/1A89f4LruLzNzHPTs19EhHRUqyTuTQ-ga/view?usp=sharing

#4

kindly provide access to this file. I have requested one also. And once you do it just drop a message here in discussion.

THank You :slightly_smiling_face:.

#5

Done. I have given viewing as well as editing permissions.

#6

hey , can you please upload your jupyter notebook on github and share me its link .
Its actually a bit fuzzy to look on the above link provided.

#7

https://drive.google.com/file/d/1HcnaqztgmHB0O1BGmHBi2-hiUeomz-AI/view?usp=sharing

#8

hey @Jalaanchal-Tewari-1816244721737412 ,
there are many cases for such performance,

  1. Most important , data. You currently have very less data. So although your model is learning well. But still it not much generalised and might perform poorer.

  2. You can taken max len as 10, where as a large number of sentences are less than 7 . Hence adding those extra zeroes will somewhat lower your performance. To counter it , you need to search about it on web.

  3. You need to work on parameter tuning like , learning rate, number of nodes in each layer , optimizer etc.
    I tried by adding another LSTM layer after the first one ,with learning rate 0.0001 and adam optimizer, and after evaluation i was getting 80% accuracy on test set.

So these are some suggestions you can use to improve performance.
I hope this helps.
Thank You :slightly_smiling_face:.

#9

I understand that I’m using very less data. But then, shouldn’t my validation accuracy be fluctuating constantly? Why is it staying constant?

I tried by changing the model architecture and playing with the model parameters but I can’t get the validation accuracy to budge.

Capture
Capture1574×650 147 KB

How did you get 80% accuracy on test set?

#10

hey @Jalaanchal-Tewari-1816244721737412 ,
have a look at my code.
https://colab.research.google.com/drive/1lZ6oDe0tAI7IoG67ktcOvaG34STNE-AC?usp=sharing

#11

I hope I’ve cleared your doubt. I ask you to please rate your experience here
Your feedback is very important. It helps us improve our platform and hence provide you
the learning experience you deserve.

On the off chance, you still have some questions or not find the answers satisfactory, you may reopen
the doubt.

1 Like
#12

Hey i got the same problem too, it’s not constant but too small and my val loss are really high. I use LSTM for my chatbot model with 500 training data.

image
image1728×721 57.1 KB

#13

Your model is getting overfitted.

You need to work a lot on your model.

  1. Add regularizations
  2. lower learning rate.
  3. add more layers. ( LSTM or other )
  4. different optimizers.
  5. change number of nodes

try these once.

#14

ok i will try adding more layers and other things

#15

hmm i already tried many things and it still not working…
i added more layers already and changed number of nodes multiple of times, i have tried different optimizers and lower my learning rate too @_@

image
image1114×687 24.4 KB

#16

share your code file , with me.
need to check this beautiful architecture.

#17

sorry for the late reply, ok wait

#18

https://drive.google.com/drive/folders/1rSyQqjrmzKjw38iSSP0OLMDCptbHFAtl?usp=sharing