LARGE LANGUAGE MODELS (LLMs)

Question 1

3 types of LLM Architecture: Transformer encoder

Answer 1

for sequence to label
eg used by BERT
(no decoder)
Allows any two states to connect
Used for POS tagging

Question 2

3 types of LLM Architecture: Transformer decoder

Answer 2

for sequence generation
eg used by ChatGPT
Prediction from previous states is the input of the current state
autoregressive: it produces a token taking into account all previous states (slower than parallel processing)
No connection to the subsequent states

Question 3

3 types of LLM Architecture: Transformer encoder-decoder

Answer 3

Hybrid
sequence to sequence
used by text to text transfer transformer

Question 4

What are Model Parameters

Answer 4

An LLM contains a large amount of parameters like input vectors for tokens (token embeddings), neural network weights and projection matrices for attention, etc

Question 5

What is the Objective of training

Answer 5

To find the best (or sufficiently good) values for the model parameters using training instances
(optimisation)

Question 6

What is the Objective Function in training

Answer 6

What we want to optimise (maximise or minimise)
eg “performance function” or “loss function”
Performance measures examined over a collection of tasks using training instances
Eg:
Whether the model can correctly predict the ground-truth output given the input instances

Question 7

What is Training

Answer 7

Finding good model parameters by optimizing the objective function O(Θ), e.g., maximizing the log-likelihood and/or minimizing the cross-entropy loss, computed over the training instances

Question 8

What is cross entropy loss

Answer 8

A loss function to approximate classification error
outputs between 0 and 1

Question 9

What is Iterative optimisation

Answer 9

Given an objective function O(Θ), one starts from a random initial guess of Θ, and iteratively applies a change to the guess to increase (or decrease) O(Θ)

Question 10

What is Mini-batch Gradient Descent for Training LLM

Answer 10

We update the parameters based on an estimated gradient of the objective function, computed over a small set (called a batch) of training instances - instead of the whole dataset

Question 11

What are hyperparameters

Answer 11

They are training parameters to determine before the training starts
not learned from the data but are rather choices made by the practitioner to configure the learning process: crucial for a good model

Question 12

What is the main two-step approach in LLM training

Answer 12

• Unsupervised pretraining using unlabelled text.
• Supervised fine-tuning based on downstream NLP tasks

The success of an LLM largely relies on its training design, such as:
- how to construct training tasks
- how to prepare training data

Question 13

What is an Epoch

Answer 13

A complete pass through the entire training dataset
During, the model sees and processes every example in the training dataset exactly once
epoch hyperparameter - how many times the algorithm will work through the entire training dataset

Question 14

What is BERT training

Answer 14

Bidirectional Encoder Representations from Transformers (BERT)
Uses a Transformer Encoder structure, taking either a single sentence or a sentence pair as an input
sentence pair: combined sequence of tokens from two sentences,
starts with [CLS] and [SEP] in between two sentences
e.g“[CLS] my dog is cute [SEP] he likes playing”.

Question 15

What is the BERT Pretraining

Answer 15

It is pretrained on two text collections:
- Book corpus
- English Wikipedia

It is pretrained on two learning tasks:
-masked language model (MLM) pretraining
-next sentence prediction (NSP) pretraining

Question 16

BERT Training: MLM

Answer 16

Text is preprocessed into tokens
Randomly selects 15% tokens, within that:
10% - keep unchanged
80% - replace with [MASK]
10% - replace with random token

Task - predict the masked tokens
If it works - trained successfully

Question 17

BERT Training: NSP

Answer 17

Want to extract sentence pairs from corpus
For each sentence:
-50% - select sentence that truly follows and label with ‘yes’
-50% - select random sentence and label with ‘no’

Binary classification task - classify an input sentence pair to yes or no

Question 18

What is BERT Fine-tuning

Answer 18

BERT is finetuned on a collection of NLP tasks that exists
The tasks are converted to classification tasks

Question 19

What is GLUE BENCHMARK

Answer 19

A collection of NLP tasks that can be used to fine tune BERT

Question 20

What are Generative Pre-trained Transformers (GPT)

Answer 20

A type of LLMs introduced by OpenAI
uses a transformer decoder structure, supported by generative pretraining and supervised fine-tuning

Question 21

What is GPT training

Answer 21

generative pretraining predicts each token from its previous k tokens (using a conditional modelling method)
The autoregressive decoder structure enables adding the predicted token to the input tokens

Question 22

What is GPT Fine tuning

Answer 22

also trained on the glue benchmark
After unsupervised pretraining
Supervised finetuning involves simply concatenating correct input and output sequence pairs, with a delimit token, e.g., “$”

The pairs are passed to the decoder to form a representation vector which can then be used in the prediction layer for specific classification tasks

Question 23

What is Instruction Fine-tuning

Answer 23

an effective way to prepare input-output pairs for training
describes all NLP tasks using natural language instructions
and fine tunes an LLM to understand and process these instructions

Question 24

What is Chain of Thought Annotation

Answer 24

Improves LLM for unseen reasoning tasks
prepares the data better by asking a human expert to annotate the reasoning of the data
Eg add an annotation to the answer to explain how it got there

Question 25

What is Learning from Human Feedback

Answer 25

using reinforcement learning with human
feedback
Over 50 experts asked to give recommendations which was fed into mitigations and improvements for the model