Convex Optimisation Flashcards

1
Q

What is a convex function?

A

A function 𝑓(π‘₯) is convex if for any two points
π‘₯1 and π‘₯2 and any 𝑑 ∈ [0,1], the inequality 𝑓(𝑑π‘₯1 + (1βˆ’π‘‘) π‘₯2) ≀ 𝑑𝑓(π‘₯1)+(1βˆ’π‘‘) 𝑓(π‘₯2) holds. Example: 𝑓(π‘₯) = π‘₯2
.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What defines a convex set?

A

A set 𝑆 is convex if the line segment joining any two points in 𝑆 entirely lies within 𝑆.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is a convex optimization problem?

A

A convex optimization problem seeks to minimize a convex function over a convex set.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

How does the gradient descent algorithm work in convex optimization?

A

Gradient descent iteratively adjusts parameters in the direction opposite to the gradient of the function at that point to find the local minimum, which is also the global minimum in convex problems.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Describe Newton’s Method in the context of convex optimization.

A

Newton’s Method uses both the first and second derivatives to find a better approximation of the minimum faster by considering the curvature of the function.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is the purpose of line search in optimization algorithms?

A

Line search determines the optimal step size in a given direction to efficiently decrease the objective function and ensure convergence.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What defines a good descent direction in optimization methods?

A

A good descent direction is identified using local information to ensure it leads to a decrease in the function value. Methods like gradient descent use the negative gradient as the descent direction.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What are common challenges faced by the gradient descent method?

A

Challenges include slow convergence on flat surfaces, getting stuck at saddle points, and the need for careful step size selection to avoid overshooting.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly