Transfer Learning Flashcards
Transfer learning
Leveraging pre-training models by reusing feature representation
Why use transfer learning
You don’t have enough computational resources to train a model from scratch.
To save time compared to training from scratch.
When you have a small training set
When may transfer learning NOT work well?
The pre-training model feature can’t differentiate the classes in your problem. You remove too many layers from the pre-trained model
General steps for transfer learning
Obtained the pre-training model.
Create a based model from the pre-trained model.
Freeze layers of the pre-trained model
What is fine-tuning in the context of transfer learning?
Unfreezing the pre-trained model and retraining at a low learning rate
Where can you find pre-trained models for computer vision tasks?
Keras applications
Stanford’s ImageNet pre-trained models
Which is typically used as a pre-trained model for NLP tasks?
Word embedding like Glove or Word2Vec & hugging face
What are three main ways to use a pre-trained model?
Prediction
Fine tuning
Feature extraction
What is the first step when using transfer learning with image data?
Get the pre-trained model
Why is it important to freeze batch normalization layers when using a pre-trained model?
To prevent the layer mean and variance from being updated
Valid data augmentation technique
a) Random rotation
b) Horizontal flipping
c) Zooming
When fine-tuning a model, what should you use? (Select all that apply)
Unfreeze the pre-trained model
Retrain at a low learning rate
Which of these is typically used as a pre-trained model for NLP tasks?
Word embeddings like GloVe or Word2Vec
When creating an embedding layer from pre-trained word embeddings, why is it important to set trainable=False?
To prevent the embeddings from being updated during training
What type of layer is commonly used for sequential data like text?
Embedding
