Chapter 1

Question 1

Natural Language Processing

Answer 1

Concerned with processing natural languages such as English and Mandarin. Involves translating natural language into data that computer can use to learn about the world.

Question 2

NLP system

Answer 2

Referred to as a pipeline because it involves several processing stages where natural language flows in one end and processed output flows the other.

Question 3

FST (Finite State Transducer)

Answer 3

FSM that outputs a sequence of new symbols as it runs is called a finite state transducer

Question 4

Formal languages

Answer 4

A Set of natural languages. Formal grammar can be used to generate many natural language statements.

Question 5

Regular expressions

Answer 5

Special kind of formal language grammar

Question 6

Regular grammars

Answer 6

Predictable, provable behavior and flexible enough to power some sophisticated dialog engines and chatbots

Question 7

DFA (Deterministic Finite Automaton)

Answer 7

A formal mathematical object that processes regular language is called a Finite State Machine or Deterministic Finite Automaton

Question 8

Regular exp notation

Answer 8

- OR
\ - preceding char can occur 0 or more times
[] - used to specify character class
* - regular expression matches any number of consecutive characters

Question 9

Computational Theory of Mind

Answer 9

CTM assumes human-like NLP can be accomplished with finite set of logical rules that are processed in series

Question 10

Distance Metrics (Levenshtein, Jaccard and Euclidean distance)

Answer 10

Useful for applications like spelling correctors and recognizing proper nouns where algorithm calculates the distances between words to find any spelling errors

Question 11

Document Representation

Answer 11

Can be represented as a vector, a sequence of integers for each word or token in that document.

Question 12

Vector space

Answer 12

Different ways that word could be combined to create vectors. Relationships between these vector make up our model, which tries to predict combinations of words occurring in a collection of various words. Can represent these vector using a Counter in python.

Question 13

Disadvantage with bag of words

Answer 13

Does not work well for interpreting context of sentences (those for which order is very important)

Question 14

Disadvantage with one-hot vectors

Answer 14

High-dimensionality space

Question 15

SyntaxNet and Spacy

Answer 15

Two libraries that allowed natural language syntax tree parsers and made possible to extract syntactic and logical relationships

Question 16

Chatbot processing stages

Answer 16

(PAGE)

1) Parse – extract features, structured numerical data from natural language text (SOTA: Tokenizers, Regular Expressions, tag, NER, extract info)
2) Analyzer – Generate and combine features by scoring text for sentiment, grammatically and semantics (Typically use a database is used) (SOTA: LSTM)
3) Generate – Compose possible responses using templates, search or language models (Search Templates, MCMC, LSTM, FSM)
4) Execute – Plan statements based on conversation history and objects and select the next response

A feedback loop (between 1 and 3) is used on generated text responses so that responses can be processed using same algorithms used to process user statements

Question 17

Layers for feature extraction and analysis

Answer 17

Characters -> Tokens -> Tagged tokens -> Syntax tree (fed into POS tagger) -> Entity relationships -> Knowledge base (fed to logical compiler, info extractor)

Question 18

Inferences

Answer 18

Logical extrapolations from a set of conditions detected in an environment

Question 19

Fuzzy regular expressions

Answer 19

Find closest grammar match among possible grammar rules instead of exact ones. Effective for question answering systems and task-execution assistant bots.