Words and Morphology Flashcards
What is lemmatization?
-
What does tokenization accomplish?
-
What is a lemma?
A lemma is the dictionary form of a word. “fly” is a lemma, it is what you would see in a dictionary, “flying”, “flies”, “flew” and “flied” all have the lemma “fly”. They are word-forms of the lemma fly and they all belong to the same “lexeme”.
What is a word-form?
The word-form is the full inflected or derived form of a word.
What is the relationship between a lexeme and a word-form?
word-forms are members of a lexeme. A lexeme is a set of word-forms
What is tokenization?
Processing text to decide/extract “words”
In terms of corpora: what is a “type” and what is a “token”, what is the difference between them?
Types are the number of distinct words in a corpus; if the set of words in the vocabulary is V , the number of types is the vocabulary size |V| Tokens are the total number of running words. In the sentence “eye for an eye”, there are 4 tokens but only 3 types. “eye” is repeated.
What does a “word” usually refer to? Types or Tokens?
Types.
What law do the number of words in a language follow?
Zipf’s law
f×r≈k
where f is the frequency of a word, r its rank among an ordered set of other words and k is some constant
What law do the number of types and tokens in a corpus follow?
“Herdan’s Law” or “Heaps’ Law”.
What is a morpheme?
The elementary unit of morphosyntax that compose word types (morphotactics).”desalination” is made up of the morphemes: de+salin+ate+ion
Give two example of categories of morpheme
stems, bound morphemes, root, pattern, reduplication and affixes
Give examples of types of affixes
suffixes, prefixes, circumfixes, infixes
What is a “bound morpheme”
A morpheme that is used to construct a word type but alone does not have meaning.In fact+u+al - “u” and “al” are bound morphemes.
What is the stem of a set of word-forms?
The part of the word-forms that they all share.–stem of walk, walks, walked, walking, walker, walkers is “walk”.
This can go as far as the stem of “produce” and “production” being “produc” which isn’t a proper morpheme or word.
What do prefixes typically do in English?
Change the meaning of words.
What is inflectional morphology?
Inflectional morphology typically–does not change basic meaning or part of speech–expresses grammatical features or relations between words–applies to all words of the same part of speech.”word” becoming “words” is inflectional morphology.
Changes the grammatical role of a word in a sentence (whether it’s the subject, object, its count etc.)
It often depends on the othe words in the sentence.
What is derivational morphology?
- may change the part of speech or meaning of a word.
- changes the word-class, i.e. a noun can become a verb or an adjective or adverb etc.
- is not driven by syntactic relations outside the word
- may be “picky”: drama+(t)ize but not traged(-y)+ize
Difference between inflectional morphology and derivational morphology
inflectional morphology does not change the meaning of a word but may affect its grammatical structure, it is dependent on the context in which the word is placed.
derivational morphology often changes the meaning of the word and the class of words it belongs to, for example a noun may become a verb, adjective or adverb.
derivational morphology applies closer to the stem whereas inflection occurs at word edges: govern+ment+s, centr+al+ize+d”ment” and “al+ize” are derivational while “s” and “d” are inflectional.
What is compounding?
Creating new words by merging existing words.
