I don’t understand how error becomes constant after some iteration , shouldn’t error denote a convex graph.How does one find out how much iterations are needed to get to a local minima?
Plotting Error_list & iterantions
Hi Shubhankit,
Even Though the error seems constant to you, it is not constant it is still decreasing but with very very small magnitude, and after many iterations it reaches it’s global minimum value, after that no matter how many more iterations you run it error will remain same. This minimum error value depends on dataset i.e it will be different for a different dataset.
The convex graph you were talking, that comes when we plot error vs theta. (either we increase or decrease theta error will increase).
A basic Question - Why do we run many iterations? why not just a single iteration everytime?
Ans - after every iteration the thetas values shift towards the optimum theta and that shift is defined by learning rate, and it happens slowly, not in just 1 iteration, which means in every iteration we are trying to reduce the error.
Therefore when we are plotting error_list vs. iterations, we are always getting decreasing graph. (After a few iterations decrease is very very small)
Now, your last question : How does one find out how much iterations are needed to get to a local minima?
In this case, since the graph of error vs theta will be a convex graph we won’t get a local minima, rather you should ask how many iterations needed to get to global minima.
For this purpose we run the model for many iterations like 300, 500… so that we can be very sure that the error has reached to global minima, and theta values are now optimized, that’s where you can see the constant line in error_list vs iterations plot.
I hope you get the point now 