Classification, with word sense disambiguation example

Question 1

Supervised machine learning

Answer 1

Learn from training data where word sense is already labelled.

Apply to test data.

Approaches:

• Bag of words
• Word n-grams

Question 2

Support vector machines

Answer 2

Interpret the feature list geometrically, and try to derive a separating hyperplane between positive and negative

Encode features as numbers and think of them as coordinates

Question 3

Naive Bayes classifiers

Answer 3

Probabilistic model

Learn

• Probability for different labels
• The probability of co-occurrence of a feature and a label

Question 4

The assumptions used to derive the probability of a sense given a feature list

Answer 4

• Bayes rule

- The assumption of independence between features

Question 5

How to compute the probability of a sense given a feature list

Answer 5

P(sense | feature list)

Using Bayes Rule, the equals
argmax P(feature list | sense) * P(sense)

Question 6

How to compute the probability of a sense

Answer 6

P(sense)

= Count (sense) / Count (all sense)

Question 7

How to compute the probability of a feature given a sense

Answer 7

Assume that all features appear independently:
P(feature list | sense)
= P(feature1 | sense) * … * P(featureN | sense)

Calculate using counts (maximum likelihood estimation)

P (“red” | well known person)
= count (label W & feature “red”)/
count (label W)