Generative AI

Question 1

RLHF

Answer 1

Reinforcement learning from human feedback

Question 1

PEFT

Answer 1

Parameter efficient fine tuning

Question 2

Self-Attention

Answer 2

• In order to predict the next word accurately, models need to be able to see the whole sentence or whole document
• The transformer architecture unlocked this ability
• Able to pay attention to the meaning of words it’s processing and attention is all you need

Question 3

Multi-headed Self-Attention

Answer 3

-Multiple sets of self-attention weights or heads are learned in parallel independently of each other
- The outputs of the multi-headed attention layers are fed through a feed-forward network to the output of the encoder

Question 4

How many parameters does a model with general knowledge about the world have?

Answer 4

Hundreds of billions

Question 5

How many parameters do you need for a single task like summarizing dialog or acting as a customer service agent for a single company?

Answer 5

Often just 500-1,000 examples can result in good performance

Question 6

Context window

Answer 6

Space available for the prompt

Question 7

Inference

Answer 7

• Generating a prediction
• For LLMs, that would be using the model to generate text

Question 8

Completion

Answer 8

Output of the model

Question 9

Entity recognition

Answer 9

Word classification to identify all the people and places

Question 10

Foundational models by decreasing number of parameters

Answer 10

Bloom -> GPT -> Flan-T5 -> LLaMa -> PaLM -> BERT

Question 11

RNN

Answer 11

• Recurrent neural networks
• Used by previous generations of language models

Question 12

What’s so important about the transformer architecture?

Answer 12

• The ability to learn the relevance and context of all the words in a sentence
• It can be scaled efficiently to use multi-core GPUs
• It can parallel process input data making use of much larger training datasets
• Dramatically improved the performance of natural language tasks over earlier generation of RNNs

Question 13

Instruction Fine Tuning

Answer 13

Adapting pre-trained models to specific tasks and datasets

Question 14

RAG

Answer 14

Retrieval Augmented Generation

Knowledge base data is used for the retrieval portion of the solution

Question 15

What’s significant about the transformer architecture

Answer 15

• Can be scaled efficiently to use multi-core GPUs
• Parallel process input data making use of much larger training datasets
• Dramatically improved the

Question 16

Origin of the Transformer Architecture

Answer 16

Attention Is All You Need

Question 17

What are attention weights?

Answer 17

The model learns the relevance of each word to all other words during training

Question 18

What are the two distinct parts of the transformer architecture

Answer 18

Encoder and decoder

Question 19

Tokenize

Answer 19

• Convert words to numbers with each representing a position in a dictionary of all possible words
• There are multiple tokenization methods. Token IDS can match two complete words or parts of words
• The same tokenizer used to train the model must be used to generate the text

Question 20

Embedding Layer

Answer 20

• Trainable vector embedding space
• High-D space where each token is represented as a vector and occupies a unique location within that space
• Each token id in the vocabulary is matched to a multi-dimensional vector
• During model training, the vectors learn to encode the meaning and context of individual tokens in the input sequence

Question 21

What was the vector size in Attention Is All You Need

Answer 21

512 dimensions

Question 22

Positional encoding

Answer 22

Position of word in sentence/document

Question 23

What is passed into the encoder/decoder

Answer 23

• Token vectors and positional encoding
• Processed in parallel

Question 24

What is passed to the self-attention layer

Answer 24

• The combined positional encodings and token vectors (single vector)
• The model analyzes the relationships between the tokens in your input sequence

Question 25

GPT

Answer 25

Generative Pre-trained Transformers

Question 26

Limitations of RNNs

Answer 26

• Limited by the amount of compute and memory needed to perform well on generative AI tasks
• Next word prediction only
• No self-attention
• Not possible to scale the models from next word prediction to self attention

Question 27

Heads

Answer 27

Attention weights

Question 28

Feed Forward Neural Network

Answer 28

• Information moves only in one direction, from the input layer through any hidden layers and finally to the output layer
• There are no cycles or loops in the network
• connections between the units do not form a cycle, unlike in RNN

Question 29

ReAct Prompting

Answer 29

• Reasoning + Acting
• LLMs are used to generate both reasoning traces and task-specific actions in an interleaved manner
• Reasoning models allow the model to induce, track, update action plans, and handle exceptions. That allows the LLM to interact with external tools to retrieve additional information

Question 30

LangChain

Answer 30

• provides AI developers with tools to connect language models with external data sources
• prompt templates