General

Question 1

What is transfer learning?

Answer 1

Technique where we use a pre-trained model which are trained on a sufficient amount of dataset to perform on a similar task that is related to what it was trained on initially, or when there is limited data available for the second task.

Question 2

What to do when target dataset is same as base model dataset?

Answer 2

Feature Extraction

Unfreeze only fully connected layers, and train it using new dataset suitable for required no. of classes.

Question 3

What to do when target dataset is different from base model dataset?

Answer 3

Fine tuning

Unfreeze fully connected layers as well as some of the layer of base model from the end, and train it using new dataset suitable for required no. of classes.

Question 4

Sequential model vs Functional model

Answer 4

Sequential model -
* 1 Input
* 1 Output
* Linear

Functional model -
* Application e.g., - Emotion and Age prediction of human
* Multi Input - Multi output model
* Non-linear
* Works using concatenation or addition

Question 5

Why was transfer learning not preferred for NLP tasks before 2018?

Answer 5

This is because of task specific job of NLP. However, with the introduction of ULMFiT (Universal Language Modelling Fine Tuning) research paper, where NLP model was trained for language modelling (which is next word prediction) as a pre-training task.

Question 6

Why is language modelling preferred as a pre training task?

Answer 6

1. Rich feature learning
2. Unsupervised task