No of units in LSTM Layer

#1

The number of units basically denote the size (length) of the internal vector states, h and c of the LSTM and it is a hyperparameter , Right ??

But do they have any kind of dependency on input sequence length we provide during training ?

Is there any way to determine what would be an optimal value for no of units ?

Suppose in my input data, no of timesteps = maxlen of any sequence = 35, and now I take a very small value say 10 as my number of units and consider another scenario where I take a large value say 512 , how will the information flow will be affected in both the scenarios and how will these values eventually affect my model performance (apart from computational complexity) ??

Taking values like 128, 256 does it mean that we are allowing higher level or basically more detailed information to flow through LSTM cells ?

#3

hey @gautam75 ,
The number of units/cell in LSTM layer define how long you need to carry you information.
Its not like that you keep a very high value , it will remember till end , no .

It actually depends upon the data that you input , and hence based on that you need to decide what values will be better.

There is no such decide as such , but yeah trying different values can help you out , or the last option parameter tuning.

As said , it depends , maybe it might learn better in any or not . larger values generally say less remembering the data.

connect it like human mind , to remember for long you need to learn those things again and again after some time to keep them in your minds memory. Same things happen with LSTM too.

Not neccessary , the main reason is the GPU cores , and computation works in power of 2 ,hence these values are more suggested , but it doesn’t say that we need to use them only.

I hope this helps a bit.

#4

I hope I’ve cleared your doubt. I ask you to please rate your experience here
Your feedback is very important. It helps us improve our platform and hence provide you
the learning experience you deserve.

On the off chance, you still have some questions or not find the answers satisfactory, you may reopen
the doubt.