[NLP] Lecture 3: Efficiency, Efficiency, Efficiency (Max Müller-Eberstein)

Question 1

What are the three pillars of efficiency?

Answer 1

-Compute: is expensive in electricity, in Denmark we are blessed with a lot of green energy
- Data (efficiency)
- Effort (how difficult is it to get started with NLP fx?)

Question 2

How many times more power does a chatgpt question take compared to a google search?

Answer 2

10 times more

Question 3

How many % do we use in denmark on data centres?

Answer 3

Almost 20%

Question 4

Where does the model live?

Answer 4

In the cloud, which lives in some hardwere. We use GPUs to train, if we have a really big model, we use more GPUs and we can also use more servers.

Question 5

Why does it work to split the model into servers?

Answer 5

When we use transformer models, we use blocks, and we can add blocks to another server, wait for the one block to finish and send data to the next sever with a block.

Question 6

Explain what a transformer is made of

Answer 6

We feeed some word-ids (word vectors) into the transformer) and this output some vector with some word probability.

Vectors -> attention head -> feedforward

In the attention, what goes on it multiplying the vector with the query, key and value vectors. These words doesnt really mean anything, we do not know what goes on

Question 7

WHat is considered a small LLM model?

Answer 7

32 blocks

Question 8

Exp

Question 9

Why is the transformer not efficient?

Answer 9

If you want to change something with the model, you need to change the whole model, because we dont know what does what

Question 10

How can we make it more efficient?

Answer 10

• Only train some parts of the network. The problem is, if we tweak something we dont know how it will effect other things.
  We can also do what is called parameter-efficient fune-tuning: adding new parameters to the model: adapter (provide a new hidden state?), prefix (pretends there is more context to the sentence), tuning and LoRA (like an adapter but at a different location)

Question 11

What do we do now

Question 12

Explain how an adapter works, how prefix tuning works and how LoRA works

Question 13

What ways can we make efficient use of the data we have?

Answer 13

• Gather as much data as your possible target language
• Cross-lingual Transfer:
• Train on all data that you can find and hope for the best
• Learning dynmaics:
• Understand how and when models learn certain things
• Also cross-lingual transfer