Syntactic Analysis: POS-Tagging

Question 1

POS Tagging

Answer 1

• Tags what part of speech each word is
• Uses POS Taggers like WordNet, Penn Bank, Brown Corpus, etc.
• Words that have more than one POS occur more often
• Eric Brill’s algorithm it has a set of rules for walking through a sentence and assigning labels. As it steps through the words, it applies the rules and it may change previous word taggings.
  1) Assign the most common POS word initially
  2) List of rules that can change the initial tag based on the words before and after the word. Set of a few hundred rules.

Question 2

Using POS tags

Answer 2

-Default tagger: assigns the same tag to each token; good for establishing a baseline

Question 3

Type vs. Token

Answer 3

Unique vocabulary word, you can have more than one of the same word in tokens. Types are unique tokens.

Question 4

Hidden Markov models (HMMs)

Answer 4

• Been around since the 1980s
• Statistical approach (stochastic)
• Baum-Welch algorithm to generate HMM from training data
• Viterbi algorithm - determines the most likely sequence of “states” of the HMM, most likely sequence of POS tags.

Question 5

Stochastic Taggers (aka statistical models)

Answer 5

• Stochastic incorporate frequency and/or probabiility
• POS probability based on the preceding word(s)
• Hidden Markov models (HMMs)
• POS with probabilities for each word, multiply them all together, use the highest scoring model of all HMMs, then tag sentence based on that model

Question 6

Dynamic programming

Answer 6

Fibonacci Series
Recursively find the answer
Fib(0) = 0, Fib(1) = 1
Transforms to a matrix, one end and go across the matrix depending on which one is most likely
Next tag depends on values of the next two tags

Question 7

Machine learning taggers

Answer 7

• N-gram trainer
• Naive Bayes
• Neural network
• SWVM

Question 8

Machine learning taggers

Answer 8

• N-gram trainer
• Naive Bayes - nbt.tag(tokens) - 93.1% accurate
• Neural network
• SVM

Question 9

Training data

Answer 9

• Penn Treebank: Over 7 million POS-tagged text made up of IBM training manuals, WSJ articles, transcribed telephone conversations
• You can build your own, e.g., if you are trying to POS tag tweets
• Could just tag based no the most common POS for that word (90% accuracy)
• Human consensus among English teach tagging WSJ articles is 96%, probably not possible to get to 100%

Question 10

Process

Answer 10

• Gather corpus
• Sentence tokenizer
• Word tokenizer
• POS tagger
• Higher Level Analysis

Question 11

Combine taggers

Answer 11

• We can combine taggers through a voting engine. This uses a panel of different taggers (NB, HMM, Brill, etc.)
• Every different POS tagger gets a vote and majority wins

Question 12

POS Tagging Use Cases

Answer 12

• Narrow down synset lookups in WordNet, also helps with synonyms/hyponyms to substitute related search terms
• Rudimentary Named Entity Recognition or Extraction (get names of proper people, places, things)
• Potentially more useful word clouds (e.g., eliminate verbs to make a word cloud less messy)
• More robust sentiment analysis, mixed sentiment review of movies

Question 13

Tropoynym

Answer 13

More specific cases of another verb. Troponym for speaking would be verbalizer, whimper, etc.