In the stemming part

[sankalparora5 (sankalp arora)]

why is that i get an error when i write
stemmed_tokens=ps.stem(new_tokens)

and this lines works perfectly,

stemmed_tokens=[ps.stem(token) for token in new_tokens]

what is the difference ??

[prashant_ml (Prashant Arora)]

@sankalparora5

stemmed_tokens=ps.stem(new_tokens)

hey @sankalparora5 ,
actually the PorterStemmer accepts input to be just a single word, but you are providing directly a list of tokens which it can handle. Hence , it doesn’t works and gives you an error.

@sankalparora5

stemmed_tokens=[ps.stem(token) for token in new_tokens]

In this command you have iterated over each token , and by providing each token individually the stemmer works fine as it gets the input it requires.

I hope this would have resolved your doubt.
Thank You and Happy Leanring .

[prashant_ml (Prashant Arora)]

I hope I’ve cleared your doubt. I ask you to please rate your experience here
Your feedback is very important. It helps us improve our platform and hence provide you
the learning experience you deserve.

On the off chance, you still have some questions or not find the answers satisfactory, you may reopen
the doubt.