Week 2: Discriminant Functions

Question 1

Discriminant Functions

Answer 1

They’re scalar functions that assign a feature vector to a class. They result in the feature space divided by decision boundaries into decision regions.

Question 2

Dichotomiser

Answer 2

A discriminant function that can only assign feature vectors to 1 of 2 classes.

Question 3

Linear Discriminant Functions

Answer 3

Discriminant functions whose decision boundaries are linear.

Question 4

Augmented Vectors

Answer 4

Vectors that contain all parameters and features

Question 5

Quadratic Discriminant Functions

Answer 5

Discriminant functions whose decision boundaries are defined by quadratic functions.

Question 6

Linearly Separable Data

Answer 6

When the dataset has classes that can be cleanly divided by linear discriminant functions.

Question 7

Sample Normalisation

Answer 7

If there are only two classes, multiply the samples in the negatively-labelled class by -1, so that samples can all be modified using the same mathematical and statistical operations.

Question 8

Gradient Descent

Answer 8

The method of updating weights and parameters using a learning rate and gradient vector.

Question 9

Batch Perceptron Learning

Answer 9

Loop through all samples in a batch, and only pay attention to the misclassified examples. The updated weights are then the old weights plus all the misclassified samples multiplied by eta.

Question 10

Sequential Perceptron Learning

Answer 10

Go through each sample, updating the weights by the misclassified sample times eta. Keep iterating until the termination criteria is reached.

Question 11

Multiclass Perceptron Learning Algorithm

Answer 11

If a sample is misclassified, move the weights of the actual class closer towards the sample and the weights of the other classes away from the sample.

Question 12

Widrow-Hoff/LMS Learning

Answer 12

A type of gradient descent where the update function is the original weight plus eta times the difference of a margin value and the weights multiplied with the input, then multiplied with the input. This ensures that the relative accuracy of the weights in classifying the samples is factored in the update function.

Question 13

k-Nearest Neighbours (k-NN) Classifier.

Answer 13

A way of classifying samples based on the majority class of the k closest samples to the specific sample in question.

Pros:
- No training time
- Classification accuracy high with large sample size
- Works with multi-modal data and non-linearly separable data
- Can use a proportion of neighbours in each class to estimate probability that a sample belongs to each class

Cons:
- Large storage requirements
- Computationally expensive
- Number of training samples required increases exponentially with dimensionality of feature space