Part 1 : Deterministic Data Models

Question 1

Data to Model

Answer 1

• Models are descriptions of the data, they encode our assumptions
• Models are a ‘generalisation’ of the data

Question 2

Choice of model

Answer 2

Often dictated by practicality of method, as well as our assumptions of the data - No need to play God

Question 3

A model aims to achieve

Answer 3

maximum discrimination

Question 4

Linear Classifier

Answer 4

Only one parameter needed

Question 5

Deterministic Models

Answer 5

produce an output without a confidence measure

- (Do not encode uncertainty in the data)

Question 6

(Deterministic Model) - Line Fitting

Answer 6

The best fitting line is that which minimises a distance measure from the points to the line. Can use the Method of Least Squares

Question 7

Method of Least-Squares

Answer 7

R(a, b) = Σ(yi − (a + bxi))2
- (This is known as the residual)
aLS = y¯ − bLSx¯
bLS = Σ (xiyi − Nx¯y¯ / xi^2− Nx¯^2 )
Note :: x¯,y¯ <= mean x,y

Method :

1. Minimise residual by taking the partial derivatives, and setting them to zero. (using the chain rule)
2. Find aLS and bLS
3. y = aLS(x) + bLS

Question 8

(Method of Least-Squares) - Outliers

Answer 8

Have a disproportionate effect, because residual is defined in terms of squares differences

Question 9

(Method of Least-Squares) - Matrix Form

Answer 9

R(a, b) =(also)= ||y − Xa||^2
where..
y = column matrix y1...yn
X = column matrix, first column, 1, second column, x1...xn
a = [a
       b]

y - Xa = [y1 -a - bx1
…. -a - ……
yn -a - bxn]

[aLS
bLS] = (((X^T)X)^-1)(X^T)y

…Look at slide 103

Question 10

K-D Least Squares - Matrix Form

Answer 10

[aLS
bLS] = (((X^T)X)^-1)(X^T)y

Where ((X^T)X) is a (K+1)(K+1) square matrix

Question 11

General Least Square - Matrix Form

Answer 11

Matrix formulation also allows least squares method to be extended to polynomial fitting.

For a polynomial of degree p + 1

yi = a0 + a1xi + a2xi^2 + …+ apxi^p

[aLS
bLS] = (((X^T)X)^-1)(X^T)y

Where ((X^T)X) is a (P+1)(P+1) square matrix

Question 12

Generalistation and overfitting - Least Square, Polynomial Lines

Answer 12

Think of how lines of different order polynomials have different shapes…. order of p has a big impact on the separation of data.

See slide 108