unconstrained optimisation

Question 1

global minimiser:

Answer 1

a point x* in R^n is a gm if for all x in R^n, f(x*)<=f(x)

Question 2

local minimiser:

Answer 2

a point x* in R^n is a lm if there is an open neighbourhood U of x* such that f(x*)<=f(x) for all x in U

Question 3

neighbourhood:

Answer 3

x* is a point in a space X, the neighbourhood of x* is a set V that includes an open set U, so x* in U within V within X, aka there’s an open ball in S with centre x*, if V is an open subset of X then it’s an open neighbourhood - look at wikipedia for visual examples basically

Question 4

strict local minimiser:

Answer 4

a point x* in R^n is an slm if there is an open neighbourhood U of x* such that f(x)<f(x) for all x in U such that x!=x

Question 5

isolated minimiser:

Answer 5

a point x* in R^n is an im if there is an open neighbourhood of x* such that x* is the only local minimiser in U

Question 6

C^(k)(u):

Answer 6

set of functions that are k time continuously differentiable on some set U

Question 7

gradient of f at x:

Answer 7

∇f(x)=((∂f/∂x1)(x),…,(∂f/∂xn)(x))^T

Question 8

f in C^1(U) with U an open neighbourhood of x, x local minimiser:

Answer 8

∇f(x*)=0

Question 9

hessian:

Answer 9

∇^(2)f(x)=∂^(2)f/∂xi∂xj, 1<=i, j<=n, this makes a nxn symmetric matrix apparently

Question 10

local minimiser and derivatives:

Answer 10

if x* is a local minimiser, f’(x)=0 and f’‘(x)>=0
if f’(x)=0 and f’‘(x)>0, x* is definitely a local minimiser

Question 11

positive semidefinite:

Answer 11

for a matrix A, A⪰0, if for every x in R^d x^(T)Ax>=0, positive definite just remove the line under the ⪰ and >

Question 12

local minimiser and ∇fs:

Answer 12

f in C^2(U) for some open set U and x* in U, if x* is a local minimiser ∇f(x)=0 and ∇^2f(x) is positive semidefinite, the converse is also true, if that holds and ∇^2f(x) is positive definite then x is a strict local minimiser

Question 13

convex set:

Answer 13

C within R^n, all x,y in C and λ in [0,1], C is convex if λx+(1-λ)y in C, convex body if C is closed and bounded

Question 14

convex function:

Answer 14

S within R^n, f:S->R^n is convex if S is convex and for all x,y in S and λ in [0,1], f(λx+(1-λ)y)<=λf(x)+(1-λ)f(y), is strictly convex if that’s just a <

Question 15

concave:

Answer 15

function f is concave if -f is convex

Question 16

if f is convex a local minimiser is:

Answer 16

a global minimiser

Question 17

f in C^1(R^n), f is convex iff:

Answer 17

for all x,y in R^n, f(y)>=f(x)+∇f(x)^(T)(y-x)

Question 18

f in C^2(R^n), f is convex iff:

Answer 18

∇^(2)f(x) is positive semidefinite, if it’s positive definite it’s strictly convex

Question 19

level set:

Answer 19

of the function f, {x: f(x)=c}, c is the level

Question 20

contour plot:

Answer 20

a plot of a collection of level sets, each different c is a different curve

Question 21

convex functions and contour plots:

Answer 21

the sub-level sets {x: f(x)<=c} are convex sets, the areas between curves I think? or under them idk