Week 1

Question 1

What is unsupervised learning? And what are some examples

Answer 1

Extracting patterns from input data with no inputted target label (ie doesn’t know

Question 2

What is supervised learning? And what are some examples

Answer 2

Minimising the error between prediction and targets (ie training knows exactly how far from optimal and what the optimal is)

Question 3

What is reinforcement learning? And what are some examples

Answer 3

Note: RL is neither supervised nor unsupervised

Question 4

What is the difference between underfitting and overfitting?

Answer 4

Question 5

What is a method involving the dataset to reduce overfitting

Answer 5

Split the dataset

Question 6

What are the 3 data splits, their usages and their typical percentages

Answer 6

(Just train val = 80:20) Val is used to select the model (ie early stopping) test is purely for performance.

Question 7

How does cross-validation work and what problem is it designed to solve

Answer 7

Question 8

What is the curse of dimensionality

Answer 8

Question 9

What is the no free lunch theorem

Answer 9

Question 10

What is the difference between parametric and non-parametric models

Answer 10

Question 11

What are the pros / cons of parametric models (and an example model)

Answer 11

Question 12

What are the pros / cons of non-parametric models (and an example)

Answer 12

Question 13

What is linear regression

Answer 13

Question 14

What is the linear regression error function and what is the closed form optimisation solution

Answer 14

Question 15

How can you use linear regression to model non-linear relationships

Answer 15

Functions to modify input data into features. Note: linear refers to the weights being linear

Question 16

How does regression work with probabilistic models

Answer 16

Question 17

What is maximum likelihood estimation used for

Answer 17

Question 18

What is the math definition of MLE

Answer 18

Question 19

What is the MLE recipe (and what is the likelihood function)

Answer 19

Question 20

How does the least squares solution relate to the MLE solution

Answer 20

Question 21

Why are probabilistic models often better than deterministic models

Answer 21

Question 22

What is classification

Answer 22

Question 23

What is a simple linear discriminant function and how does it relate to classification

Answer 23

Linear discriminant can be considered to be the decision boundary (ie line/hyperplane)

Question 24

What is linear separability

Answer 24

Question 25

What is logistic regression and how does it work

Answer 25

Question 26

How does MLE work for logistic regression

Answer 26

Gradient descent. No closed form

Question 27

How does MLE using gradient descent work

Answer 27