NLP Flashcards
What is TRUE for Natural Language Processing?
improves human-computer communication = TRUE
improves human-human communication = TRUE
improves computer-computer communication = FALSE
distills knowledge from texts TRUE
ELIZA
- was meant to be a parody on a Rogerian psychoanalyst
- programmed by J. Weizenbaum (1966)
- works by a very simple pattern matching on responses
- has no understanding of the conversation
- still one of the best-known AI chatbots
- Weizenbaum’s intention WAS NOT to demonstrate AI
How does ELIZA work?
- Puts sentences into keywords
- Keywords have rank/precedence numbers
- Commas/Periods delimiters?
- Analyzes input acc. to transformation rules by decomposing sentences
- keyword + transformation rules = script
- Response is generated by reassemly rules ass. to decomposition rules (e.g., replacing I with YOU)
If ELIZA gets stuck, what does it do?
- returns to keywords from prio convo
- has a few stock answers
- is sensitive to specific subjects (like family)
Cross three properties which are true about ELIZA!
- can only respond with a question about the current input = FALSE
- scans input sentence for keywords = TRUE
- is not able to return to previous content =FALSE
- uses stock answers sometimes = TRUE
- uses artificial neurons, i.e. a neuronal net = FALSE
- can be sensitive to specific subjects (e.g. subject family) = TRUE
PARRY
counter-part to ELIZA
imitates a paranoid schizophrenic
takes advantages of giving silly responses
it passed a restricted Turing TEST in 1970
STUDENT
- was able to solve simple High School math problems
- but you had to type your question in a natural language
- viewed every sentence as an equation
- used trigger words to identify the task
SHRDLU
- natural language interface to the block’s world
- could perform tasks, give name to objects, memorize operations & answer questions about the state of the world.
Match the chatbots:
SHRDLU = NL interface to the block’s world
PARRY = passed a restricted Turing test
ELIZA = responses are based on reassembly rules
STUDENT = could solve high school math problems
Tokenization is a big problem in NLP.
TRUE
Word Segmentation/Tokenization
- dividing the input text into small semantic entities
- issues are z.b. New York vs. “New” and “York” one token or two tokens?
- numbers have different formats
- also different abrbreviation rules (US vs U.S.)
Part-Of-Speech-Tagging (POS)
- Each word in a sentence can be assigned to a category
- noun, verb, adjective etc.
- based on the definition of the words (thesaurus)
- POS uses definitions (thesaurus) and context (grammar rules) to decide the category
- can be formulated as a sequence learning task such as Hidden Markov Models (HMMs) or Conditional Random Fields (CRFs)
What is NOT a fundamental NLP task?
lexical analysis = TRUE
syntactic analysis
semantic analysis
word segmentation
Modern NLP systems use knowledge based MT systems instead of Deep Learning.
False
The key idea behind Word2Vec is that any word - represented as a vector will be mapped to
familiar words.
True
Which of the following is advantage of using a system like Word2Vec?
capable of capturing syntactic and semantic relationships between different words = FALSE
can have out-of-vocabulary words as well = TRUE
effort for humans to tag data was less, because it is a unsupervised technique = TRUE
vector size is not direct proportional to vocabular size = TRUE
Machine Translation
- has been studied early on
- very hard because of the ambiguity
- Neural Machine Translation allows end-to-end learning
- input is the raw INPUT TEXT
- Encoder = a neural network maps the input into the intermediate representation (–> embedding)
- Decoder = another neural network maps the embedding into a different language text
- Results are much better than previous system, but still not perfect
Transformer Networks
BERT = Convolution
GPT = Recurrence
NLP
= maps the query to a structured representation of the content
3 out of 4 points are reasons why Natural Language Processing (NLP) is hard. Which point does not match the others?
The grammar is very complex for NLP = TRUE
Natural languages are semantically and in a discourse very ambiguous.
There are often hidden meanings that are not obvious from the message itself. (jokes, puns, sarcasm, …)
Natural languages are lexically and syntactically highly ambiguous.
Neural Machine Translation allows end-to-end learning. In which order?
- Intermediate representation (embedding)
input of the raw input text (not grammatical analysis) - Encoder: a neural network maps the input into an
- Decoder: another neural network maps the embedding into a different langua- ge text
- Results typically better than previous systems (but still not perfect)
The basic idea of ”Information Retrieval” is: a document is regarded as a vector in an n- dimensional space and is a linear combination of the base vectors. Linear algebra can be used for various computations.
True
The definition (after Grishman 1997, Eikvil 1999) of ’Information Extraction’ is: ”The identification and extraction of instances of a particular class of events or relationships in a natural language text and their transformation into a structured representation (e.g. a database).”
True
Natural Language Processing (NLP) distill knowledge from texts in different ways. Ther are ”information retrieval” (IR) and ”information extraction” (IE)
a) IR retrieves relevant documents from collections (e.g., search engines).
b) IE retrieves relevant information from documents (e.g., comparison shoppers).
True
The ”Bag of Words Model” assumes that the document has been generated by drawing once a specified number of words (without replacing them) out of a bag of words.
False
Which sentences about ”Bag of Words Model” are correct?
Each word in the bag can appear once. Each word is therefore drawn with the same probability. = FALSE
It’s like drawing letters out of a Scrabble-bag (without replacement).
The ”Bag of Words Model” assumes that the document has been generated by drawing
once a specified number of words out of a bag of words. = FALSE
It’s like drawing letters out of a Scrabble-bag, but with replacement. = TRUE
Words in the bag may occur multiple times, some more frequently than others. So each word is drawn with a different probability. = TRUE
”Bag of Words Model” assumes that the document has been generated by repeatedly drawing one word out of a bag of words. = TRUE
It’s about the intuition of ”Information Retrieval” or ”The Vector-Space Model”.
Documents that are ’close together’ in the vector space talk about the same things.
True
Semantic Analysis has traditionally two levels (Representing meaning of individual words and
Word sense disambiguation)
Nowadays Semantic Analysis will be often done via deep learning (semantic embeddings). There are semantic embeddings of words (e.g., Word2Vec) and of sentences (e.g., Bert)
True
Key Idea of ”Word2Vec” (a semantic embadding) is to find a distributed word representation, i.e., each word is represented as a lower-dimensional, non-sparse vector. This allows, e.g., to compute cosine similarities between words. The general approach is to train a (deep) neural network in a supervised way using the context of a word as additional input.
True
Is the following statement true?: Weizenbaums NLP-Model ”Eliza” already had a deep understanding of the conversations it held.
False
Match the following descriptions to their associated term in NLP.
- Lexical = same word, different meanings
- Discourse = meaning of sentence might depend on previous sentence
- Syntactic = same sentence, different interpretations
- Semantic = sentence interpretation may depend on context
Word Segmentation
Tokenization
Syntactic Analysis
- Parsing
- grammatical analysis of a sentence to determine which word is in which part of the sentence
- 1) S = NP VP
- 2) NP = Det N + PP
- 3) VP = V NP or VP PP
- 4) PP = P NP
- Marx-Brother’s Joke = Elephant and PJ’s
Semantic Analysis
-Semantic Embeddings
-Finding a representation of an utterance (word, phrase, sentence, document) in Euclidean space
-Distance in this space captures similarity in meaning
-Utterances that are close to each other have similar meanings
-Traditional Approach -> Bag of Words, where each word has one dimension
- Nowadays semantic embeddings are done via deep learning
* of words in Word2Vec
* of sentences in Bert**
WHY is NLP so hard?
NLP is highly ambiguous
Lexical:
The same word may have different meanings like zamok vs zamok
Syntactic:
The same sentence may have different interpretations. See: “The chicken is ready to eat”.
**Semantic: **
The interpretaion of the sentence may require a deeper understanding.
**Discourse: **
The meaning of the sentence depends on the prio sentences in the text.
1st Sentence:
“John told Paul he made a mistake.”
2nd Sentence:
“He apologized immediately.”
Hidden Meanings: like sarcasm, jokes etc.
Match the words to their definitions in terms of why NLP is difficult:
Syntactic = the same sentence may have different interpretations
Discourse = the meaning of a sentence also depends on the previous sentences in the conversation/text
Semantic = the interpretation of a sentence may depend on its content
Lexical = the same word may have different meanings
What is true for the ”Bag of Words” model?
-Different classes have different bags of words = TRUE
-Normally, bags do not contain high context depending words because they mostly don’t appear that often = TRUE
-Assumes document is created only with words from one subset of the vocabulary = FALSE
-Words in the bags can occur multiple times depending on their usual quantity of usage
in this context = TRUE
The simple Na¨ıve Bayes classifier for a text uses the probability p(ti/c) with which the word ti = wi occurs in the document class c.
True
What is NOT true about PARRY, the chatbot?
passed a restricted Turing test
was able to give absurd responses
was the predecessor of ELIZA = CORRECT
attempted to behave like a paranoid schizophrenic
