Text Classification

Question 1

What are the possible uses of text classification? (3)

Answer 1

Spam Detection

Topic Classification

Sentiment Analysis

Question 2

Formalise the text classification objective.

Answer 2

Given a document (d) and a set of classes C. We want to assign the document d to the most appropriate class c.

Question 3

What is the Naive Bayes assumption?

Answer 3

features are conditionally independent given class.

Question 4

What are the features of a document for topic classification?

Answer 4

Usually the count of the most frequent words

Question 5

What are the features of a document for sentiment analysis?

Answer 5

Usually the count of words from a sentiment lexicon. Some words are attributed to certain classes. For example “good”, “adorable”,brave” would be associated with a positive class while “bad”, “ugly”, “cowardly” would be associated with a negative class.

Question 6

How is the prior estimated for naive bayes?

Answer 6

Normally done using maximum-likelihood estimation. The number of documents that belong to a class C over the total number of documents is the prior for class C - P(C)

Question 7

How are the conditional probabilities of the features in text classifcation for naive bayes estimated?

Answer 7

Normally done using smoothing applied to maximum-likelihood estimation.

Question 8

Alternative features for text classification

Answer 8

Use binary features (did this word x occur in the document, yes or no). Use only a subset of the vocabulary. Use more complex features (morphological features, bigrams, synctatic features).

Question 9

Advantages of Naive Bayes

Answer 9

1) Fast and easy to train/test 2) Simple model so is easy to implement 3) Doesn’t require as much training date 4) Usually works well

Question 10

Disadvantage of Naive Bayes

Answer 10

The naive independence assumption is very weak, words tend to be correlated. So features are not really independent.

Question 11

What is intrinsic evaluation?

Answer 11

An evaluation measure inherent to the task

Question 12

Given an example of an intrinsic evaluation measure for language modelling

Answer 12

Perplexity

Question 13

Given an example of an intrinsic evaluation measure for POS tagging

Answer 13

accuracy (% of tags correct)

Question 14

Given an example of an intrinsic evaluation measure for categorization

Answer 14

F-score

Question 15

What is extrinsic evaluation? Give an example for language modelling

Answer 15

measure effects on a downstream task Language modelling: Does it improve my ASR/MT task?

Question 16

How to deal with unbalanced classes?

Answer 16

1) Collect more data 2) Augment some of the data you do have 3) Create copies of training samples.

Question 17

What is the precision?

Answer 17

items the system detected that were right/items the system detected

In other words (true positives)/(false positives + true positives)

Question 18

What is the recall?

Answer 18

Items the system detected that were right/items the system should have detected

(true positives)/(true positives + false negatives)

Question 19

What is the F-measure (equation)

Answer 19

F_β = ((β²+1)PR)/(β²P + R)

Question 20

What is the harmonic mean of the precision and recall? (F-1)

Answer 20

The F-measure with β set to 1: F₁ = 2PR/(P + R)

Question 21

How is the F-measure decreased/increased

Answer 21

For precision and recall values that are close to eachother, the F-measure is smaller. For those that are further apart, the F-Score is much bigger.