Classification

Question 1

Describe the steps of Image classification

Answer 1

1) Feature extraction
2) Feature description
3) Classification

Question 2

How can we classify images without labels

Answer 2

Use unsupervised clustering techniques

Question 3

What do we mean by semi-supervised learning?

Answer 3

We only have labels for a part of the dataset

Question 4

What is MNIST

Answer 4

A dataset of 70 000 28x28 pixel handwritten digits

Question 5

Describe the typical pre-processing of digit recognition in general images

Answer 5

1. Detect the digits in the large image
2. Normalize the size of the digit, for example, to 28x,28 pixels
3. Normalize the location, place mass center in the middle
4. “Slant” Make the orientation canonical

Question 6

Describe K-Nearest Neighbour algorithm

Answer 6

Classify by taking a majority vote among the K-nearest neigbhours

Question 7

Name some distance measurements

Answer 7

L2 (Euclidean), L1(Manhatten)…

Question 8

Name some advantages and disadvantages of K-Nearest Neighbour

Answer 8

1. It works reasonably well
2. No training required
3. Nonlinear decision boundaries
4. Multi-class
5. All training data must be stored in memory
6. Long evaluation time

Question 9

Why shouldn’t we use the test set to tune hyperparameters? What should we do instead?

Answer 9

Tuning hyperparameters with the test set overfits the model to the test set. Use a validation set or cross-validation.

Question 10

What’s the advantage and disadvantage of using cross-validation vs. a dedicated validation set

Answer 10

Cross-validation makes more data available for training but is computational more expensive.

Question 11

What’s the formula of a general linear classifier

Answer 11

w*x + b = y

Question 12

how can we use the formula of a linear classifier, 
wx + b = c to assign a class to the data?

Answer 12

in class (+1) if c>0, else not in class (-1)

Question 13

What is considered the best hyperplane for SVMs?

Answer 13

The hyperplane that maximalizes the margin, the combined distance from the closest points in both classes to the hyperplane.

Question 14

What points do we need to determine the hyperplane in an SVM?

Answer 14

Only the points closest to the hyperplane

Question 15

Derive the formula for making the margin equal to 2 in an SVM.

Answer 15

𝒘 ∙ 𝒙𝟏 + 𝑏 = -1
𝒘 ∙ (𝒙𝟏 + 𝑚𝒏) + 𝑏 = 1.
m 𝒘 ∙ 𝒏 = 2
𝒏 = 𝒘/||𝒘||

Question 16

How can we deal with outliers in SVM’s

Answer 16

We can introduce a slack variable which allows some misclassified points. This will increase the margin.

Question 17

Why can we always solve the minimization problem of the SVM, assuming that the classes are linearly separable?

Answer 17

The loss is convex

Question 18

How can we solve the minimization problem of the SVM loss in practice

Answer 18

1. Gradient descent and the subgradient

2. Lagrangian duality

Question 19

Why do we use a kernel in SVMs?

Answer 19

Transforming the data to a higher dimension might make it linearly separable.

Question 20

Name some SVM kernels

Answer 20

Linear, Polynomial, Gaussian.

Question 21

Name some strategies for multi-class classification

Answer 21

1 vs. Rest

1 v 1 with a majority voting

Question 22

How large is the slack variable in an SVM when a point is on the wrong side of the separating hyperplane

Answer 22

slack > 1