Topic 5: Word Sense

Question 1

recap on word ambiguity

Answer 1

same word can be used to mean different things.

“mouse”

• small rodent
• hand-operated device to control a cursor

“bank”

• hold investments in a custodial account
• river bank

is called polysemous..from Greek word meaning having many senses.

Question 2

what is word sense?

Answer 2

discrete representation of one aspect of the meaning of word

Question 3

Wordnet

Answer 3

this is an online thesaurus. database that represents word senses

Question 4

Word sense disambiguation

Answer 4

task of determining disambiguation which sense of a word is being used in a particular context.

Question 5

Homonym and orthographic

Answer 5

example bank1 and bank2 have same orthographic form but sense are unrelated

in this case, it’s homonym.

if the words are homonym, it means the words that have the same spelling and pronunciation but different meanings and origins.

Question 6

Homograph

Answer 6

words that are not necessarily pronounce the same and having different meanings and origins.

bow1 and bow2

Question 7

Homophones

Answer 7

each of two or more words have same pronunciation but different meaning, origins or spelling.

Question 8

Dictionary or thesauruses

Answer 8

document/database that give textual definition for sense called glosses.

Question 9

Dictionary

Answer 9

contain many fine-grained senses to capture meaning differences

Question 10

Glosses

Answer 10

not a formal meaning representation. is written for people

Question 11

Sentence embedding

Answer 11

sentences have glosses along to help build sense representation.

Question 12

Relation between sense: synonymy

Answer 12

two sense of two different words which are identical or nearly identical

synonymy is a relationship between senses rather than words

example
count/sofa
vomit/throwup
car/automobile

Question 13

Relation between sense: antonym

Answer 13

words with an opposite meaning
example:
long/short
big/little
fast/slow
cold/hot
dark/light
rise/fall
up/down
in/out

Question 14

Taxonomic relations: hyponym

and hypernym

Answer 14

One sense is a hyponym of another sense if the first sense is more specific

example
car is hyponym of vehicle
dog is hyponym of animal
mango is hyponym of fruit

hypernym is the other way around

superordinate is often used instead of hypernym
superordinate - subordinate

Question 15

meronymy

Answer 15

part-whoe relation

example

leg is part of chair

wheel is part of car

Question 16

Wordnet

Answer 16

English WordNet consists of nouns, verbs, adjectives-adverbs

8 sense for noun “bass”

usually have a gloss, synonym set and usage example

Question 17

Synset

Answer 17

set of near-synonyms for a Wordnet sense..

a way of representing concept

synsets are the fundamental unit associated with Wordnet entries.

also labels synset with lexicographic category drawn from a semantic field.

Question 18

Sense relation in wordnet

Answer 18

WordNet has two kinds of taxonomic entities: classes and instances

Question 19

Verb relation in Wordnet

Answer 19

shows the relation such as hypernym, troponym, entails, antonym

Question 20

Hyponymy Chain

Answer 20

Hyponymy chains for two
separate senses of the
lemma bass.

Note that the chains are
completely distinct, only
converging at the very
abstract level whole, unit

Question 21

Thesaurus Method

Answer 21

Use the structure of the thesaurus to define word similarity.

Can use any information from gloss to synonym etc.

In practice, hypernym/hyponym hierarchy is used

the intuition is
Words or senses are more similar if there is a shorter path
between them in the thesaurus graph.

Question 22

Path Length

Answer 22

Measure the number of edges between
the two concept nodes in the thesaurus
graph and adding one

pathlen(c1,c2) = 1 + the number of edges in
the shortest path in the thesaurus graph
between sense nodes c
1 and c2.

Question 23

Path-Length based Similarity

Answer 23

path-length based similarity equation . ….

For most application, we do not have sense-tagged data, hence word
similarity algorithm gives similarity between words, taking the
maximum sense similarity

word similarity equation . . .

Question 24

Information-Content Word Similarity

Answer 24

Rely on the structure of the thesaurus but also add probabilistic
information derived from a corpus

Define P(c) as the probability that a randomly selected word in a corpus
is an instance of concept

P(root) = 1, since any word is subsumed by the root concept.

Intuitively, the lower a concept in the hierarchy, the lower its
probability.

Question 25

Thesaurus with Probability

Answer 25

A fragment of the WordNet concept hierarchy augmented with the
probabilities P(c)

Question 26

Information Content Theory

Answer 26

Need two more definitions for the similarity computation, Information
Content Theory (IC) and Lowest Common Subsumer (LCS)

equation
IC(c) = - logP(c)

Question 27

Lowest Common Subsumer

Answer 27

LCS(c1,c2) = the lowest node in the hierarchy that subsumes both c1 and
c

Question 28

Resnik Similarity

Answer 28

Think of similarity between two words as related to their
common information.
sim resnik 𝑐𝑐1, 𝑐𝑐2 = −logP(𝐿CS(𝑐𝑐1, 𝑐𝑐2))

Question 29

Word Sense Disambiguation recap

Answer 29

the task of selecting the correct sense for a word

WSD algorithm takes the input of a word in context and a fixed
inventory of potential word senses and output the correct word sense
in context

Question 30

WSD Datasets

Answer 30

The inventory of sense tags depends on the task.

need to make sure it is from the same domain

use set of sense from resource like WordNet or supersense if want a coarser-grain set

Question 31

Baselines for WSD Systems

Answer 31

A surprisingly strong baseline
is simply to choose the most frequent sense for each word from
the senses in a labeled corpus

Question 32

Supervised Word Sense Disambiguation

Answer 32

Labeled dataset - context sentences labeled with the correct sense for
the target word

can use any standard classification algo

as for the feature
- collocation features of words or n-grams of lengths 1, 2, 3
- Bag of word – words that occur in the neighborhood
- Weighted average of embeddings
- Part-of-speech tags (for a window of 3 words on each side,
stopping at sentence boundaries)