Natural Language Processing

Question 1

What is a Natural Language Processor (NLP)?

Answer 1

An NLP is a processor with the goal of making machines understand and interpret human language.

Question 2

What is meant by linguistic analysis?

Answer 2

Analysis that should be undertaken before language processing to help the machine understand the formation of words and their relationships.

Question 3

What is the linguistic analysis of syntax?

Answer 3

An analysis of what part of the text is grammatically correct.

Question 4

What is the linguistic analysis of semantics?

Answer 4

An analysis of the meaning of the given text.

Question 5

What is Natural Language Understanding (NLU)?

Answer 5

A Natural Language Understanding (NLU) module attempts to understand the meaning of a text. This means that the nature and structure of each individual word in the next must be known.

Question 6

How does a Natural Language Understanding module understand the structure of a sentence?

Answer 6

By resolving ambiguities present in the natural language.

Question 7

What is a lexical ambiguity?

Answer 7

The multiple meanings of words.

i.e. Polysemy: ‘bank’ - capital bank or a river bank.
Synonymy: ‘big’ and ‘large’ have identical meanings.

Question 8

What is a syntactic ambiguity?

Answer 8

The multiple parse trees, or logic, of a sentence.

Question 9

What is a semantic ambiguity?

Answer 9

The multiple meanings of a sentence.

i.e. The cat is sleeping on the couch. The couch is where the cat is sleeping.

Question 10

What is an anaphoric ambiguity?

Answer 10

A phrase or word that refers to an entity mentioned previously in the sentence.

i.e. Lucy went to the cinema. She had fun.
‘She’ refers to ‘Lucy’, the entity mentioned prior. This is important information to understand the second sentence.

Question 11

What is syntax analysis?

Answer 11

The analysis of sentence structure, including parts of speech (POS).

Question 12

What is semantic analysis?

Answer 12

The analysis of the meanings of words and phrases.

Question 13

What is Named Entity Recognition (NER)?

Answer 13

The identification of named entities in the input.

Question 14

What is intent recognition?

Answer 14

The understanding of the speaker’s intent.

Question 15

What are the four steps of natural language understanding?

Answer 15

Syntax analysis: structure
Semantic analysis: meaning
Named Entity Recognition (NER): entity identification
Intent Recognition: understanding

Question 16

What are the primary steps of the NLP pipeline?

Answer 16

Segment individual sentences using punctuation. Identify different words, numbers and punctuation by tokenizing. ‘Stem’ the words by stripping the ending (i.e. ‘ending’ turns to ‘end’). Assign each word a ‘tag’ that designates the word as either a noun or adverb. Divide the next into different categories. Identify entities, and define the relationship between certain words and the entities of previous sentences.

Question 17

What is sentence segmentation?

Answer 17

The method of separating a paragraph into its individual sentences. This is typically done using punctuation marks.

Question 18

What is tokenisation?

Answer 18

Tokenisation is the technique of separating a sentence into a list of their words, numbers and punctuation.

Question 19

What is stemming?

Answer 19

Stemming is the removal of the ‘endings’ of words, or their suffixes by simply chopping them off.

i.e. ‘ending’ -> ‘end’, or ‘sprinting’ -> ‘sprint’.

Question 20

What is lemmatization, and how is it different from stemming?

Answer 20

Lemmatization is the ‘proper’ removal of word endings compared to stemming.

While stemming simply chops the words off, lemmatization attempts to remove only inflectional endings and return words to their ‘dictionary form’.

i.e. The word ‘meeting’ can either be the base form of a noun (a meeting), or a form of a verb (to meet). Stemming would always produce ‘meet’, while lemmatization could distinguish between the two.

Question 21

What are stop words?

Answer 21

Stop words are the most common words in a language.

i.e. ‘and’, ‘the’, ‘a’, etc.

Question 22

What is Part-of-Speech (POS) tagging?

Answer 22

POS tagging is a supervised learning solution to tag words using features such as the previous word, next word, capitalization etc. to distinguish it as one of the eight ‘parts of speech’.

Question 23

What is the Bag of Words model?

Answer 23

The bag of words model is a feature extraction technique that simply describes how often a word appears in a document.

Question 24

How can we use Bag of Words to analyze a document?

Answer 24

We can assume that two documents with a similar meaning will use some of the same words. Comparing the occurrence of words between documents allows us to get a base idea of what the original document is about.

i.e. We could identify a document as a literature review by comparing its contents to other literature reviews.

Question 25

What is the purpose of the TF-IDF rating?

Answer 25

TF-IDF is a statistical measure used to evaluate the importance of a term to a document.

It considers how often the term appears in that document - Term Frequency - and the number of documents with that term in it - Inverse Term Frequency.

Question 26

What is word prediction?

Answer 26

The use of the probabilities of sequences of words to calculate the likeliest next word. This is typically done through MLE or Chain Rule.

Question 27

What is Chain Rule?

Answer 27

Chain Rule is a method used to calculate the probability of a sequence of words. It considers that probability as the product of the probability that Wn will occur given Wn-1, Wn-2, …, W1.

i.e. P(W1, W2, W3) = P(W1) * P(W2|W1) * P(W3|W2W1)

Question 28

What is the Markov Assumption?

Answer 28

The Markov Assumption is an assumption we make when calculating word probability.

We assume that only prior local context affects the next word. We do this to avoid the scaling problem we face when dealing with large documents.

Question 29

What is an N-gram?

Answer 29

An N-gram is an application of the Markov Assumption where we assume that the probability of a word only depends on the previous n-1 words.

An N-gram where N is 2 is called a BIGRAM. An N-gram where N is 3 is called a TRIGRAM.

Question 30

What is the benefit to using N-grams?

Answer 30

In large documents, sometimes 1000s of words long, we can ensure that the word chain is only 4 elements long, reducing the computation time.

Question 31

What are some limitations to using N-grams?

Answer 31

The higher the N, the better the model. However, the higher the N, the higher the computational overhead.

Furthermore, they are a sparse representation of language - it will simply give a probability of 0 to all words not in the training corpus.

Question 32

What is the Maximum Likelihood Estimate (MLE)?

Answer 32

MLE is a way of calculating the probability that a word will occur given a prior sequence.

It is equal to the number of times the resulting sequence exists divided by the number of times that sequence does not exist.