11 - LLMs

Question 1

WHat is attention?

Answer 1

Transformers learn which words are important (associated) to each other word

Question 2

Attention Is All You Need

Answer 2

The 2017 Google Brain Paper

Question 3

How do neural nets process words? Word Embedding

Answer 3

Transform words into vectors for feeding into neural networks

Question 4

Common Word Embeddings

Answer 4

Word2Vec, Glove and FastText

Use between 50 and 300 values to represent each word.

Similar words, similar vectors

Question 5

Why is word embedding useful?

Answer 5

Two similar words with similar meaning have a similar vector.

A network performing sentiment analysis will find it easier to learn that sentences such as “I’m happy” or “I’m cheerful” have similar sentiments

Question 6

Multi-head attention

Answer 6

Input is processed using three different learned matrices, W-Query, W-Keys and W-Values

Only contains linear operations;

Feedforward with non-linear activation is required.

Multiple versions allow for different attention mapping

Question 7

Encoder part

Answer 7

Understands and extracts relevant info from the input. Outputs a continuous representation (embedding)

Question 8

LLM Evaluation metrics

Answer 8

• ROUGE
  Used for text summarisation and compares a summary to one or more reference summaries.
• BLEU score
  Used for text translation and compares to human generated translations

Question 9

LoRA

Answer 9

Low Rank Adaptation

Question 10

LoRA: Main Idea
(Hint: freeze, parameters, weights)

Answer 10

• freeze the weights of the self-attention module
• add task-specific knowledge using a small set of tunable parameters

Question 11

LoRA Steps

Answer 11

1. Freeze most of the original LLM weights
2. insert 2 rank decomposition matrices
3. Train the weights of the smaller matrices

Steps to update model for inference:
1. Matrix multiply the low rank matrices
2. Add to the original weights

Question 12

Soft Prompts

Answer 12

Like word encoding but instead of words, are vectors that are tuned to improve model performance.

Same length as token vectors and typically 20-100 tokens

Question 13

Soft Prompts with LoRA

Answer 13

We can have different sets of prompts for different tasks.

(Switch out soft prompt at inference time to change task)

Question 14

Vanilla FIne TUning

Answer 14

Select a subset of parameters to tune and leave the rest frozen

Question 15

Reparametrisation

Answer 15

Add knowledge with additional parameters as in LoRA

Question 16

Additive fine tuning

Answer 16

Add information to the prompt with soft prompts that have been tuned for specific tasks

Question 17

Overall name for parameter fine tuning

Answer 17

PEFT - Paramter Efficient FIne Tuning

Question 18

Prompt Engineering

Answer 18

DIscovering the best format, keywords and structure sentence of the queries to produce best output

Question 19

LLM complete development and deployment pipeline

Answer 19

Understanding the software frameworks and all the steps required to create and deply an LLM for a specific application

Question 20

Reinforcement Learning with Human Feedback

Answer 20

Understanding how to learn models that can assess the output and use those models to fine-tune LLMS using reinforcement learning.