W9 Meta-Learning Flashcards
what is the difference between meta-learning and machine learning?
machine learning learns parameters that approximate the function and meta-learning learns hyperparameters about the learning-function
What is the reason for the interest in meta-learning and transfer learning?
Speed up learning a new task using previous knowledge, instead of learning from scratches.
In transfer learning, we pretrain our parameter network with knowledge from a single task. In meta-learning, we use multiple related tasks.
what are foundation models?
they are large models in a certain field (eg. image recognition or NLP) that are trained extensively on large datasets. They contain general knowledge that can be specialized for a certain purpose
What is transfer learning?
Re-use information learned in earlier training tasks from a closely related problem. Transfer of previously learned knowledge from one task to another
What is meta-learning?
learning how to learn
How is meta-learning different from multi task learning?
In multi-task learning, more than one task is learned from one dataset. The tasks are often related, such as classifcation tasks of different, but related, classes of images.
In meta-learning, both datasets and tasks are different, but not too different. A sequence of datasets and learning tasks is generalized to learn a new (related) task quidckly. The aim is learning to learn
Zero-shot learning aims to identify classes that it has not seen before. How is
that possible?
Attribute-based zero-shot learning uses separate high-level attribute descriptions of the new categories, based on categories previously learned in the dataset.
Eg. recognize a red beak because we have learned the concepts “red” and “beak”
what is few-shot learning?
test if a learning algorithm can be made
to recognize examples from classes from which it has seen only few examples in
training. Prior knowledge is available in the network.
Is pretraining a form of transfer learning?
yes: some network layers are copied to intialize a network for a new task, followed by fine tuning, to improve performance on the new task, but with a smaller dataset
Can you explain learning to learn?
Are the initial network parameters also hyperparameters? Explain.
What is an approach for zero-shot learning?
Attribute-based zero-shot learning uses separate high-level attribute descriptions of the new categories, based on categories previously learned in the dataset.
Eg. recognize a red beak because we have learned the concepts “red” and “beak”
As the diversity of tasks increases, does meta-learning achieve good results?
For tasks that are related, good results are reported, but where tasks are less related (such as pictures of animals from very different species), results are reported that are weaker.
As the diversity of tasks increases, does meta-learning achieve good results?
For tasks that are related, good results are reported, but where tasks are less related (such as pictures of animals from very different species), results are reported that are weaker.
what is the connection between meta-learning and curriculum learning?
Both approaches aim to improve the speed and accuracy of learning, by learning from a set of subtasks.
So curriculum learning is a form of meta-learning where the subtasks are ordered from easy to hard, or, equivalently, meta-learning is unordered curriculum learning
