Vector Space Model

Question 1

Vector Space

Answer 1

Defined by a linearly independent set of basis vectors.

Question 2

Orthogonal base vectors

Answer 2

v.w = 0 so v and w are orthogonal. If a set of vectors is pairwise orthogonal then it is linearly independent.

Question 3

Terms as Basis Vectors

Answer 3

Terms chosen as orthogonal base vectors but clearly not orthogonal due to polysemy and synonymy.

Question 4

Zipf’s Law

Answer 4

The frequency of a word is reciprocally proportional to its frequency:

freq(word_i) = 1/(i^theta) * freq(word_1)

Question 5

Term Importance and Zipf’s law

Answer 5

Zone 1 - High frequency words are function words so not important

Zone 2 - Mid-frequency words are best indicators of document contents.

Zone 3 - Low frequency words are generally typos or overly specific words.

Question 6

Term Frequency

Answer 6

Monotonic function of number of times a term appears in a document. (enhances recall)

Question 7

Inverse document frequency

Answer 7

Monotonic function of the number of documents in which a term appears. Use instead of inverse collection frequency as a term may have a high collection frequency by being concentrated in a small set of documents.

Question 8

Term similarity metrics

Answer 8

Terms can be compared using a string similarity metric such as edit distance.

Question 9

Determining multi-word terms

Answer 9

Observe combinations in a large corpus of text then extract multi-word terms on frequency of occurrence.