Exam 2017

Question 1

What is the primary difference between “supervised” and “unsupervised” learning?

Answer 1

whether the training instances are explicitly labelled or not

Question 2

Indicate which of “numeric”, “ordinal”, and “categorical” best captures its type: blood pressure level, with possible values {flow, medium, high}

Answer 2

ordinal

Question 3

Indicate which of “numeric”, “ordinal”, and “categorical” best captures its type: age, with possible values [0,120]

Answer 3

numeric

Question 4

Indicate which of “numeric”, “ordinal”, and “categorical” best captures its type: weather, with possible values {clear, rain, snow}

Answer 4

categorical

Question 5

Indicate which of “numeric”, “ordinal”, and “categorical” best captures its type: abalone sex, with possible values {male, female, infant}

Answer 5

categorical

Question 6

Describe a strategy for measuring the distance between two data points comprising of “categorical” features

Answer 6

Hamming distance OR cosine similarity OR jaccard OR dice

Question 7

What is the relationship between “accuracy” and “error rate” in evaluation?

Answer 7

accuracy = 1− error rate

Question 8

With the aid of a diagram, describe what is meant by “maximal marginal” in the context of training a “support vector machine”.

Answer 8

the width of the margin (= distance from separating hyperplane and the support vectors) should be maximised

Question 9

What makes a feature “good”, i.e. worth keeping in a feature representation? How might we measure that “goodness”?

Answer 9

good = correlation/association with category of interest (and non-redundant)

Question 10

For the mode below state whether it is canonically applied in a “classification”, “regression” or “clustering” setting: multi-layer perceptron with a softmax final layer

Answer 10

classification

Question 11

For the mode below state whether it is canonically applied in a “classification”, “regression” or “clustering” setting: soft k-means

Answer 11

clustering

Question 12

For the mode below state whether it is canonically applied in a “classification”, “regression” or “clustering” setting: multi-response linear regression

Answer 12

classification

Question 13

For the mode below state whether it is canonically applied in a “classification”, “regression” or “clustering” setting: logistic regression

Answer 13

classification

Question 14

For the mode below state whether it is canonically applied in a “classification”, “regression” or “clustering” setting: model tree

Answer 14

regression

Question 15

For the mode below state whether it is canonically applied in a “classification”, “regression” or “clustering” setting: support vector regression

Answer 15

regression

Question 16

With the aid of an example, briefly describe what a “hyperparameter” is.

Answer 16

a top-level setting for a given model (which is set prior to training)

Question 17

With the use of an example, outline what “stacking” is.

Answer 17

combining the output of a number of base classifiers as input to a further supervised learning model

Question 18

What is the convergence criterion for the “EM algorithm”?

Answer 18

convergence of maximum log-likelihood to within an episilon (small) change

Question 19

Outline the basis of “purity” as a form of cluster evaluation.

Answer 19

what proportion of instances in the cluster correspond to majority class

Question 20

What is the underlying assumption behind active learning based on “query-by-committee”?

Answer 20

disagreement between base classifiers indicates that the instance is hard to classify, and thus will have high utility as a training instance

Question 21

“Random forests”

are based on decision trees under different dimensions of “randomisation”. With reference to the following toy training dataset, provide a brief outline of two (2) such “random processes” used in training a random forest. (You should give examples as necessary; it is not necessary to draw the resulting trees, although you may do so if you wish.)

Answer 21

1. random sampling of training instances (similarly to bagging) 2. random subsampling of attributes for given decision tree 3. random construction of new features based on linear combinations of numeric features

Question 22

HMMs

In the “forward algorithm”, α_t(j) is used to “memoise” a particular value for each state j and observation t. Describe what each α_t(j) represents.

Answer 22

the probability of observing all observations up to and including t and ending up in state j

Question 23

HMMs

In the “Viterbi algorithm”, two memoisation variables are used to describe for each combination of state j and observation t:

1. β_t(j) (which plays a similar role to α_t(j) in the forward algorithm)
2. φ_t(j). Describe what each φ_t(j) represents.

Answer 23

he most probable immediately preceding state for state j given observations up to and including t

Question 24

HMMs

Why do we tend to use “log probabilities” in the Viterbi algorithm but not the forward
algorithm?

Answer 24

Viterbi based on multiplication, so can convert to sum of log probabilities; forward based on sum of product of probabilities,
so logging the probabilities doesn’t help in the calculation

Question 25

Model Learning

Is our primary objective in machine learning to derive a model that fits the subset of the data
that we do have? Why or why not?

Answer 25

No. Want to build a model that generalises to new data

Question 26

Model Learning

Explain how we can use our limited data, in a machine learning context, to demonstrate
whether or not our objective has been met.

Answer 26

Split data into training/dev/test. Need to measure generalisation from “best” (tuned) model to unseen data

Question 27

Model Learning

Identify and explain one important problem that can emerge with respect to this primary
objective, even if we are successful in deriving a good model for the data that we have.
Name one specific technique discussed in class that can be applied to mitigate this problem,
and explain how it does so.

Answer 27

overfitting; model does well on training data but poorly on test data

technique: L1/L2/Lasso regularisation
how: reduce complexity of model/constrain model function

Question 28

Model Learning

Define “bias” and “variance”, indicating how we might detect each one. Discuss how bias
and variance relate to each other in the context of our primary objective

Answer 28

bias: how well our model approximates the (training) data; approximation error (high bias=consistently poor performance)
variance: how well our model generalises to held-out (test) data; estimation error (high variance=sensitive to training data)
Relationship: Increasing the complexity of our model tends to reduce bias (by fitting the data better) but increase variance
(because the model will be more strongly fit to the training data); in order to obtain generalisation we seek to minimise both
bias and variance in order to have a good model that generalises well (isn’t overfit to training). Increase training examples
and control for overfitting to lower variance

Question 29

Briefly explain — in at most two sentences — the basic logic behind the “ID3” algorthmic
approach toward building decision trees. This should be focussed on labelling the nodes
and leaves; you do not have to explain edge-cases.

Answer 29

Recursively determine which feature has the highest information gain (i.e. does the best job of partitioning the data into pure
subsets) over the subset of training instances selected by the path to that node, and add a branch for each value of that feature;
continue until every leaf node is pure (and label with corresponding class)

Question 30

The criterion for labelling a node, as explained in the lectures, was based around the idea of
“entropy” — what does entropy tell us about a node of a decision tree?

Answer 30

how skewed (“pure”) the label distribution is (lower entropy ! more skewed ! better)