12 - Continual/Lifelong Machine Learning

Question 1

Traditional machine learning dev cycle

Answer 1

1. First trained on suitable dataset
2. System performance is assessed on a test benchmark
3. System is frozen and deployed in an application scenario
4. If conditions change the system is decommissioned

Question 2

Problem with traditional AI systems after deployment

Answer 2

They do not improve after deployment.

• Test data is prepared: balanced classes, interleaved learning, static distributions
  ^ This may not happen in the real world

Question 3

Catastrophic Forgetting

Answer 3

When you learn further teasks, you forget previous ones.

Question 4

Replay based methods (forgetting)

Answer 4

Relay old samples from previous tasks while learning new.

• Learn from online data and keep examples of new clases as they show.
• Interleave current data with data from old tasks

Question 5

Replay based methods challenges

Answer 5

• How do we set the balance between old and new tasks
• Memory requirements as the data can grow a lot

Question 6

Regularisation methods (forgetting)

Answer 6

Start from the idea that forgetting comes from important parameters being changed while learning new tasks

• Estimate which are important for task 1
• Devise a loss function that penalises change to those
• Learn new tasks using such a loss fn to limit interference with old tasks

Question 7

Regularisation methods challenges

Answer 7

How to estimate the importance of parameters

Most parameter could become important so it’s difficult to learn

Question 8

Parameter Isolation (forgetting)

Answer 8

A big net might require a small subset of parameters to solve a task.
Find and isolate those for a task.

• Find subset and isolate
• Grow net with additional params for new tasks
• Create masks (light binary networks for each task that select sub regions of a network that are useful)