Handout #11 - Large Language Models

Question 1

What are large language models?

Answer 1

Large transformer networks, with billions of weights anbd trained on all text available on the internet.

Question 2

What method do LLM’s rely on?

Answer 2

Auto-regressive models -> models that predict from the pas.

Question 3

Explain tokenisation

Answer 3

This is when words are mapped into vectors.

The english language is too big -> reduce size to something small

Question 4

GPT-3 uses ‘Reinforcement learning from human feedback’ (RLHF). Explain what this is

Answer 4

1. Humans are asked to rank (lots) of results.
2. Ranking are then used to score outputs.
3. Score used to train a reward/preference model
4. Fine-tune the original model through a reinforcement learning update.

Question 5

Explain the concept of jail-breaking and prompt injection attacking in LLMs

Answer 5

This is the concept of asking harmful questions through other means -> role-playing. ‘Ignore previous instructions.’

Question 6

Explain the concept of zero-shot or few-shots in LLMs

Answer 6

One can just ask LLMs to describe a task, without any example

Question 7

In terms of LLM scaling laws, what determines the performance of LLMs

Answer 7

The number of parameters in the network N and the size of the training set D.

Question 8

Give the definition of Artifical General Intelligence

Answer 8

The threshold where an agent can accomplish any intellectual task that human beings or animals can perform.