FineTuningEvaluatingModels Flashcards
____ will change the weight of the base foundation model.
Fine-tuning
The process of adapting a copy of a foundation model with your own data is called ____.
fine-tuning
Training data for fine-tuning a foundation model must adhere to a specific ____ and be store in ____.
format / S3
You must use “____” to use a fine-tuned model which is pay-per-use.
provisioned throughput
True/False: All foundation models in Amazon Bedrock can be fine-tuned.
____-based fine tuning improves the performance of a pre-trained FM on domain-specific tasks.
Instruction
What are domain-specific tasks in the context of training models?
A model is futher trained on a particular field or area of knowledge.
Instruction-based fine-tuning uses ____ examples that are ____ pairs.
labeled / prompt-response
When you want to continue pre-training a model, you need to provide ____ data to continue the training of a FM.
unlabeled
To make a model an expert in a specific domain, you must perform ____ fine-tuning.
domain-adaptation
Feeding the entire AWS documentation to a model to make it an expert on AWS is an example of ____ fine-tuning.
domain-adaptation
When performing continueed pre-training on a model, it is good to feed the model ____ terminology.
industry-specific
____ messaging is a subcategory of instruction-based fine-tuning that contains an array of message objects, each containing a role and content field.
Single-Turn
____ messaging is a subcategory of instruction-based fine-tuning for conversations, such as chatbots.
Multi-Turn
With Multi-Turn messaging, you must alternate between “____” and “____” roles.
user / assistant
Re-training an FM requires a ____ budget because it requires more computations.
higher
Instruction-based fine-tuning is usually ____ as computations are less intense and the amount of data required is usually less.
cheaper
True/False: Running a fine-tuned model is more expensive because you have to use provisioned throughput.
t
When fine-tuning a model, you must prepare the ____, do the fine-tuning, ____ the model.
data / evaluate
____ is the broader concept of re-using a pre-trained model to adapt it to a new related task.
Transfer Learning (fine-tuning is a type of transfer learning)
Transfer learning is widely used for ____ classification and ____ processing.
image / natural language
A chatbot designed with a particular persona or tone, or geared towards a specific purpose is a a use case for ____.
fine-tuning a model
Training using more up-to-date information that what the language model previously accessed is a use case of ____.
fine-tuning a model
Training with exclusive data (e.g. your historical emails or messages, internal records) is a use case of ____.
fine-tuning a model
____ are curated collections of data designed specifically at evaluating the performance of language models.
Benchmark datasets
Benchmark datasets are helpful to measure ____, ____ and ____, ____.
accuracy, speed, efficency, scalability
Some benchmark datasets allow you to very quickly detect any kind of ____ and potential ____ against a group of people.
bias \ discrimination
True/False: You can create your own benchmark dataset that is specific to your business.
t
____ evaluation of a model is where users compare model generated answers to benchmark answers.
Human
____ evaulation of a model is where a “judge” model automatically compares the benchmark answers to the model generated answers.
Automatic
