lecture 5: least squares and linear regression Flashcards
what is a function
a relation that associates each element x of a set X, the domain of the function, to a single element y of another set Y, the codomain of the function
what kinds of arguments can scalar and vector functions have?
both can have both scalar and vector arguments
what does a scalar-valued function of d-vectors do?
it maps real d-vectors to real numbers
a linear function has to satisfy which 2 properties
homogeneity and additivity, or in other words, superposition
what does homogeneity mean
scaling the vector argument is the same as scaling the function value
what does additivity mean
adding vector arguments is the same as adding the function values
a linear function is affine if and only if
it can be expressed as f(x) = aᵀx + b for some d-vector a and some scalar b, b is called the offset
graphically, an affine function will have a
y-intercept
f(x) has a local minimum at x = c if
f(x)≥f(c) for every x in some open interval around x = c
what is an open interval
it is an interval that does not include its endpoints, denoted with parentheses
the global minima is
the minimum value among all the local minima
what does argmax f(a) do
it returns the element of set a that maximises f(a)
differentiating a scalar function w.r.t. a d x 1 vector results in a
d x 1 vector
differentiation of a vector function of size h x 1 with respect to a d x 1 vector results in a
h x d matrix
what is the purpose of a learning objective function
to find the optimal values of w and b which minimises ∑(f(x) - y)²
the loss function in this case is squared error loss, and the cost function is the average squared error losses
