Gradient Descent Flashcards
How to find out the intercept with Gradient Descent?
-You have to know the slope.
-What you now do is use a random point for the intercept (in this example 0). Then you calculate the Sum of Squared Residuals(the Residual is the difference between the observed point(the point whoch is there) and the predicted point(the point on the line). You take the sum of all and have the SSR)
- Put the SSR on the y-axis. This is the intercept for x=0
If you now increase the intercept to 0.25, the SSR goes lower and more to the right. Increase the intercept, it goes again lower and to the right, and so on, until on its lowest point it increases again. At the end we have a form like a V with a round bottom.
What does the Gradient Descent do?
It takes big steps when it is far away from the optimalization (the point in the V graph where it is the lowest) and babysteps when it is close. Thus it is effective.
