Background 1 Flashcards
Terminology and Basic Math
What is global optimizer(minimizer) and optima(minimum) ?
A point x* is a global minimizer if f(x*)≤ f(x) for all x, where x ∈ D(domain) D = R^n
The value of f(x*) is the global minimum.
What is local optimizer(minimizer) and optima(minimum) ?
A point x* is a local minimizer if there is a neighbourhood N of x* such that f (x*) ≤ f(x) for all x ∈N.
The value of f(x*) is the local minimum.
what is a strict local minimizer ?
A point x∗ is a strict local minimizer (also called a strong local minimizer) if there is a
neighbourhood N of x* such that f(x) < f(x) for all x ∈ N with x ! = x.
What is convex set(region) ?
A set S ∈ R^n is a convex set if the straight line segment connecting any two points in S lies entirely inside S.
Formally, for any two points x ∈ S and y ∈ S, we have
αx + (1 − α)y ∈ S, for all α ∈ [0, 1]
What is convex function ?
The function f is a convex function if its domain S is a convex set and if for any two points x and y in S, the following property is satisfied:
f( αx + (1 − α)y ) ≤ αf(x) + ( 1 − α )f(y), for all α ∈ [0, 1]
explain continuously differentiable
we call a function continuously differentiable if it is differentiable and the derivative is continuous.
First-order necessary conditions:
If x* is a local minimizer of f and f is
continuously differentiable in an open neighbourhood of x*,
then ∇f (x*) = 0.
Second-order necessary conditions:
If x* is a local minimizer of
f and f is
twice continuously differentiable in an open neighbourhood of x∗,
then ∇f (x) = 0 and ∇2f (x*) is positive semidefinite.
Second-order sufficient conditions:
If f is twice continuously differentiable in an open neighbourhood of x,
∇f (x) = 0, and ∇^2f (x*) is positive definite at x∗,
then x* is a (strict) local minimizer of f .
what is gradient ?
Gradient is the slope of the graph of the function.
What is Hessians ?
The hessian of a function f of n variables is the matrix of second partial derivatives
