Jay Alammar

Question 1

How do we transform words into things that the Blocks can work with?

Answer 1

Words are tokenized (integer ID)
Each token is converted into its embedding (a large vector corresponding to its token integer ID)

Question 1

What are the two components of an Encoder/Decoder block?

Answer 1

Self-attention
Feed Forward Neural Network

Question 2

What’s the name for the scores given to each token in the vocabulary?

Answer 2

Logits
(these are then soft-maxed in the final layer, where they sum up to 1)
We can pick the highest one, or sample to give a wider selection from which to choose from

Question 3

Picking the highest probabilty for the next word is known as …

Answer 3

Greedy sampling

Question 4

In Self-Attention what are three matrices that are used to manipulate the inputs?

Answer 4

Query
Key
Value

Question 5

What’s the difference in architecture between GPT and Bert?

Answer 5

GPT - decoders
Bert - encoders

Question 6

What is auto-regression (in the context of LLMs)?

Answer 6

After each new token is generated, it gets added to the input prompt (ready for the next token to be generated)

Question 7

What’s the difference between Encoder and Decoder stacks?

Answer 7

They both have Self-Attention and Feed Forward Neural Network

In addition, the Decoder has **Encoder-Decoder Self Attention **

Question 8

What is length of embedding vector in GPT-3?

Answer 8

Largest version is 12,288 (dimension of vector)

Question 9

When a token is sent to a Transformer (in GPT) how is it processed? two steps

Answer 9

1. Embedding vector is looked up
2. To this is added the position encoding vector

this is then passed onto the first Decoder block

Question 10

Describe what Self-Attention does? give an example

Answer 10

Its purpose is to create a vector based on the current token, but modified for context

E.g. “the chicken crossed the road and then painted it”

each word will get a self-attention score, so “the” and “road” will score highly, and the current word “it”. Each of these vectors is then scored and the sum produces a vector which is then passed onto the FFNN (Feed Forward Neural Network)

Question 11

What is the list of tokens and their probabilities, output from the model, called?

Answer 11

Logits

Question 12

How can we consider only the top 5 probabilty words?

Answer 12

Set top_k to 5