Lecture 15 Flashcards
What can we tell about our gradient descent if the slope of the tangent line is positive?
the minimum must be to the left of the point
How to we get to the minimum when the slope of the tangent line at f(t) is positive?
decrease t
What can we tell about our gradient descent if the slope of the tangent line is negative?
the minimum must be to the right of the point
How to we get to the minimum when the slope of the tangent line at f(t) is negative?
increase t
How do we begin to minimize f(t)?
start with an initial guess t_0
What is the equation to update t and reach the minimum?
t_1 = t_0 - df/dt (t_o) where the derivative will go in the opposite direction
What is alpha?
the learning rate or step size
What is t_0?
the initial guess
What is the equation for using the update rule?
t_i+1 = t_i - alpha df/dt (t_i)
What is convergence?
when t doesn’t change much
What is the formal definition of gradient descent?
a numerical method for finding the input to a function f that minimizes the function
Why is it called gradient descent?
the gradient is the extension of the derivative to functions of multiple variables
What is a numerical method?
a technique for approximating the solution to a mathematical problem, usually using a computer
What is gradient descent widely used in?
machine learning, to train models from linear regression to neural networks
What is a convex function?
one where there is only one minimum
