Class 11

Question 1

one hot vector

Answer 1

assign a value to every word, could be used to describe every word

Question 2

word embedding

Answer 2

slightly more sophisticated approach of assigning a low dimensional vector, learned automatically from the data, do good job of representing words in isolation

Question 3

character level model

Answer 3

alternative to word embeddings – input is a sequence of characters, each encoded as a one hot vector – model has to learn how characters come together to form words

Question 4

language model

Answer 4

probability distribution over sequences of words, need to create one that has sufficient context

Question 5

machine translation

Answer 5

has the goal of translating a source language to a target language

Question 6

sequence to sequence model

Answer 6

neural network architecture created by using an RNN in conjunction with an LSTM, most commonly used for machine translation (MT) but can also be used to generate a text caption for an image or summarization – 3 shortcomings: nearby context bias, fixed context size limit, slower sequential parsing

Question 7

decoding

Answer 7

process of generating target words from source words

Question 8

transformer architecture

Answer 8

architecture that uses a self attention mechanism that can model long distance context without a sequential dependency

Question 9

multiheaded attention

Answer 9

can be used to address the problem of too much self attention – breaks sentences into pieces and applies the attention model to the pieces

Question 10

positional embedding

Answer 10

technique used by a transformer to capture the ordering of words

Question 11

transformer encoders

Answer 11

used to text classification tasks

Question 12

transformer decoders

Answer 12

used for text classification tasks but uses a version of self attention where each word can only attend to words before it