Machine Learning

Question 1

Supervised ML

Answer 1

deal with samples, provide label, learn from set of labelled samples

Question 2

classification

Answer 2

set of possible labels is finite, discrete and categorical output

Question 3

binary classification

Answer 3

has two possible labels (+1/-1)

Question 4

multi-class classification

Answer 4

has more than two (finitely) many labels

Question 5

regression

Answer 5

set of possible labels is infinite and output is continuous

Question 6

batch learning

Answer 6

given training set of labelled samples, work out labels for the samples from a test set

Question 7

online learning

Answer 7

see sample, work out predicted label, check true label carry on

Question 8

training/exploration stage

Answer 8

analyse training set

Question 9

exploitation stage

Answer 9

apply hypothesis to test data

Question 10

deduction induction transduction triangle

Answer 10

• data -> hypothesis: induction
• hypothesis -> unknown: deduction
• data -> unknown: transduction

Question 11

IID assumption

Answer 11

independent identically distributed

labelled samples (xi, yi) are assumed to be generated independently from the same probability measure

Question 12

feature

Answer 12

attribute / component of the dataset which represents a sample

Question 13

label

Answer 13

categorises the sample into a certain class, the thing we’re trying to predict

Question 14

conformal prediction

Answer 14

given a training set and test sample, try in turn each potential label for the test sample

for each label we look at how plausible the augmented training set is under IID assumption

use p-value for this

Question 15

conformity measure

Answer 15

evaluate how conforming/well the new observed test data fits with the existing observed training data, give an equivariant conformity score

Question 16

p-value

Answer 16

evaluate implausibility of augmented training set

Question 17

non conformity measure

Answer 17

similar to conformity measure but measures how strange(non-conformal) the test data is compared to training data

Question 18

average false p-value

Answer 18

the average of all p-values for all postulated labels in the test set except for the true labels

Question 19

training accuracy

Answer 19

accuracy on training set

Question 20

generalisation accuracy

Answer 20

how well the model is able to accurately predict on the test set after training on the training set

Question 21

overfitting

Answer 21

when the model learns all of the details and noise of the training set that it is not able to generalise on the test set, so it negatively impacts performance on the test set.

high training accuracy low generalisation accuracy

Question 22

underfitting

Answer 22

when the model does not capture enough of the underlying patterns of the training data due to it being too simple so cannot make accurate prediction on test data

low training accuracy low generalisation accuracy

Question 23

learning curve

Answer 23

plot of the accuracy vs size of training set n
use: understanding CV

Question 24

RSS

Answer 24

residual sum of squares
aim to minimise residuals
residuals is difference between true and predicted label

Question 25

feature engineering

Answer 25

including derived features to the training set at will for a more accurate prediction of the model

Question 26

TSS

Answer 26

total sum of squares
sum of squared difference of true label minus mean of true labels

Question 27

R^2

Answer 27

1 - RSS / TSS

% of variability in the label explained by data

low value - not compatible with poor performance on test data due to possible overfitting

Question 28

regularisation

Answer 28

each feature should have as little effect on the outcome as possible to avoid overfitting

Question 29

α regularisation parameter

Answer 29

α = 0: ridge regression coincides with least squares, no penalty applied, size of coefficients doesn’t change, keeps model complex -> overfitting

α -> ∞: coefficient estimates forced to shrink towards 0, shrinkage penalty applied, RSS less important, model too simple may lead to underfitting

Question 30

Lasso

Answer 30

Least absolute shrinkage and selection operator
-L1 penalty instead of L2/euclidean norm
-minimises RSS
sets many w[j] coefficients to 0
-LASSO performs model selection = sparse model involve only some of the features
- if a few of my features important then use this
-useful for importance of the interpretability of the model

Question 31

method chaining

Answer 31

concatenation of method calls

Question 32

data normalisation

Answer 32

measuring the dataset on the same scale to ensure compatibility of the model

Question 33

normalisation - least squares

Answer 33

not essential
if first feature x[0] measured in metres, w^[0] will be the corresponding Least Squares estimate
if instead x[0] is now measures in km, all xi[0] will decrease 1000 fold
if you run Least Squares on the new dataset, then w^[0] will increase 1000 fold so no change to predictions

Question 34

normalisation - ridge/lasso

Answer 34

essential
due to the presence of penalty terms that are the same for all variables, so predictions will change

normalising features prevents larger features from being unfairly penalised by the penalty terms

Question 35

StandardScaler

Answer 35

for each feature mean 0 standard deviation 1
1) shift each feature down by its mean
2) divide each feature by its SD

Question 36

RobustScaler

Answer 36

for each feature median 0 IQR 1
1) shift each feature down by its median
2) divide each feature by its IQR

Question 37

MinMaxScaler

Answer 37

shift each feature so it is a value ranging between 0 and 1

Question 38

data snooping

Answer 38

when test set is used for developing the model.

test set leaks into model
inaccurate normalisation
leads to data snooping
affect transformation of data
lead to overfitting/underfitting

Question 39

Normalizer

Answer 39

each sample divided by its Euclidean norm

Question 40

parameter selection

Answer 40

split training set (used for model checking) further into validation set where we select the best parameters to evaluate on test set

Question 41

inductive conformity measure

Answer 41

A : Z* x Z -> R
A(C~, z) says how well z conforms to C~
no analogue of the equivariance requirement

Question 42

kernel

Answer 42

a function that turns linear problems into non linear one
take a feature mapping F . X -> H of X = sample space into H = feature space, equipped with dot product, allows this feature mapping to be turned to K(x, x’) = F(x) . F(x’)

Question 43

kernel trick

Answer 43

• write the algorithms so that all xs can only appear in dot products
• replace the dot products with kernels

Question 43

kernel features

Answer 43

-symmetric K(x, x’) = K(x, x’)
-positive definite ∑i=1∑j=1(aiaj)K(xi, xj) ≥ 0

Question 44

decay factor lambda

Answer 44

used to weigh gaps between the substrings

dimension of feature space, value of such coordinate depends on how frequently and compactly the sustring is embedded int the text

Question 44

c

Answer 44

length of subsequences taken into account

Question 45

activation function

Answer 45

np.tanh

function nicely mapping the real line R to (-1,1)

Question 46

separating hyperplane

Answer 46

in the p-dimnsional space R^p, a flat affine subspace of dimension p-1

separates two classes

Question 47

linear scoring function

Answer 47

a function that models the hyperplane for some samples in a p-dimensional space. it separates 2 classes
if less than 0 = negative
if more than 1 = positive

Question 48

margin

Answer 48

the shortest perpendicular distance from each of the training samples to the separating hyperplane

Question 49

maximum margin hyperplane / optimal separating hyperplane

Answer 49

the farthest perpendicular distance to the hyperplane from the training samples.

Question 50

maximum margin classifier

Answer 50

classifying a test sample based on which side of the maximum margin hyperplane it lies in

Question 51

support vectors

Answer 51

vectors in the p-dimensional space R^p that lie closest to the maximum margin hyperplane

if they moved slightly the mmh moves as well

equidistant

distance between support vectors and hyperplane = slab, larger slab = greater confidence

lie on the directly on the soft margin or the wrong side

Question 52

soft margin classifier

Answer 52

the hyperplane that almost separates the classes using a soft margin

shorter margin but greater robustness and classification

soft bc it violates some of the training observations

solution to optimisation problem slide 8:46

Question 53

slack variables

Answer 53

allow individual training samples to be on the wrong side of the margin or hyperplane

Question 54

tuning parameter C

Answer 54

determines number and severity of violations to the margin and hyperplane that we tolerate

C = ∞ ->no violations tolerated, slack variables must be 0, old

C = 0 -> prioritise maximising margin, tolerate all violations

Question 55

Pipeline

Answer 55

glues multiple processing steps into a single scikit-learn estimator

fit = train model using training data, through transforming data then fitting svm
score = evaluare on test data

Question 56

cross-conformal predictor

Answer 56

≈ as full conformal predictor but
p(y) = (all ranks + 1)/n+1

calculate conformity score
rank scores
rank - 1
repeat for all folds
add all ranks
add 1
divide by n + 1 for p-value
repeat for all postulated labels

point prediction = highest p-value label
confidence = 1 - lower p-value
credibility = highest p-value

Question 57

set predictor

Answer 57

predictor that outputs prediction sets rather than point predictions and takes a significant level as parameter

Question 58

calibration curve

Answer 58

error rate vs significance level