Gradient Descent

Question 1

What is gradient descent?

Answer 1

It is an optimization algorithm used to minimize a function, the function we are talking about is the lost function ( we defined it in the previous lesson).
It is fundamental for training ML models .
The formula is a bit of a mess so we view it on iPad

Question 2

What is the main hypothesis we use when we compute gradient descent,

Answer 2

The function has to be differentiable , (in stochastic process we are not able to calculate the derivative)

Question 3

Why is it usefull the taylor series in computing the gradient descent?

Answer 3

Can be used to approximate the derivative which are computanionally intensive to calculate

Question 4

What is the main definition of gradient descent

Answer 4

So the gradient is the generalization of the derivative in more variables

Question 5

What is the consequence of considering **alpha ** too large or too small in the gradient descent

Answer 5

If alpha is too large we have a time of convergence faster but it isn’t guarantee while in the other case the convergence is surely going to happen but in a slow time

Question 6

Write down the formula and explain it

Answer 6

So the formula is messy to so I would do this exercise on paper + explaining

Question 7

Given a function f(x,y), x0, alpha ; write down the iteration to do (exam question)

Answer 7

Resolution exercise on paper