NLP

Question 1

what is NLP + examples

Answer 1

designs algorithms to allow computers to “understand” natural language to perform tasks that are useful

• language translation, making appointments, spell checks, sentiment analysis, chatbots

Question 2

steps : NLP Tasks (hint: there is 8)

Answer 2

1. Sentence segmentation
2. Word tokenization
3. Predicting parts of speech for each token
4. Text Lemmantization
5. Eliminating stop words
6. Dependency parsing, finding noun phrases
7. Named Entity Recognition
8. Coreference Resolution

Question 3

what is step 1 of NLP tasks

Answer 3

Sentence segmentation
- break paragraph into individual sentences
- easier to understand each sentence separately

Question 4

what is step 2 of NLP tasks

Answer 4

Word tokenization
- split sentence into individual words

Question 5

what is step 3 of NLP tasks

Answer 5

Predicting parts of speech for each token
- matching London(noun), is(verb), the(determiner), etc

Question 6

what is step 4 of NLP tasks

Answer 6

Text lemmatization
- identify base form of each word
- eg. “pony” is the base form for “ponies”

Question 7

what is step 5 of NLP tasks

Answer 7

Eliminating stop words
- removing common words (eg. is, the, and)

Question 8

what is step 6 of NLP tasks

Answer 8

Dependency parsing
- find out how words in the paragraph relate to each other

Finding noun phrases
- group together the words that represent a single idea or thing

Question 9

what is step 7 of NLP tasks

Answer 9

Named Entity Recognition
- Detect and label nouns wit real would concepts they represent (eg. geographic entity, person, organisation)

Question 10

what is step 8 of NLP tasks

Answer 10

Coreference Resolution
- associating pronouns with corresponding nouns
- eg. London …. It…. ( “It” == London)

Question 11

what is WordNet and problems associated with it

Answer 11

a large lexical database of English
- Nouns, adjectives, adverbs are grouped into sets of cognitive synonyms (synsets), each expressing a distinct concept

problems:
- requires humna labor to create and be up-to-date
- hard to compute accurate word similarity
- problem with rule-based or grammar-based NLP

Question 12

Word Vector Representation + problems/solution

Answer 12

words represented by one-hot vectors
- vector dimension = number of word in vocabulary

problem: no natural notion of similarity for one hot vectors
solution: represent words based on their surrounding words - have a lower dimensional dense vector rather than a one hot vector; word embedding or word vector

Question 13

Vector Space Models + examples

Answer 13

represents words by their CONTEXT
- when word appears, context is the set of words that appear nearby (fixed size window)

eg. count based methods, predictive methods, evaluating performance

Question 14

Applications of word vectors

Answer 14

• find other similar words
• find associations
• add word vectors to get vector for a paragraph
• feed word vectors to deep learners to accomplish complex NLP tasks

Question 15

Similarity between Vectors (How?)

Answer 15

Dot Product : high value = high similarity
Cosine distance : similar = 0
Cosine similarity : similar = 1

Question 16

what are Count based Methods

Answer 16

Employ co-occurrence counts
- frequencies of words occurring in same documents or sentences or within some window of each other
- each time 2 words appear within a window of each other in a sentence

Question 17

how do Count based Methods work

Answer 17

Input corpus of sentences

Step 1: Compute co-occurrence matrix, X over the entire vocabulary of corpus
- X contains counts of occurrence together for different words

Step 2 : Compute the Singular Value Decomposition (SVD) of the co-occurrence matrix
- reduce dimensions to desired level (desired size of word vector)
- X = USV^T

Question 18

what are Predictive Methods

Answer 18

1. Continuous Bag of Words (CBOW)
2. Skip gram

Question 19

Skip gram

Answer 19

• uses WORD to PREDICT target context
• shallow neural network to represent the probability model
• input: one hot representation of the center word
• output: probability of a word being in the content
• loss function: cross entropy or softmax loss
• 1 input, 1 hidden and 1 output layer
• output layer employs softmax to get probability
• NO activation function for hidden layer
• 0 bias for hidden, output layer nodes
• weights in matrices W and W’

refer to slides for math working