Optimizing Foundation Models

Question 1

The most common algorithms used to perform similarity searchs for vector storing

Answer 1

1) k-nearest neighbors(k-NN) or cosine similarity

Question 2

AWS vector database options

Answer 2

1) Amazon OpenSearch Service (provisioned)
2) Amazon OpenSearch Serverless
3) pgvector extension in Amazon Relational Database Service (Amazon RDS) for PostgreSQL
4) pgvector extension in Amazon Aurora PostgreSQL-Compatible Edition
5) Amazon Kendra

Question 3

What is RAG

Answer 3

Retrieval Augmented Generation

Question 4

What are agents? What are the types

Answer 4

Agents are autonomous computer programs that perform tasks. Main types are 1) intermediary operation agents 2) Actions launch agents 3) feedback agents

Question 5

What is the quantitative evaluation of generative AI model?

Answer 5

Benchmarking datasets. Examples of measurements include accuracy, speed and efficiency and scalability

Question 6

Instruction tuning

Answer 6

A type of fine tuning that involves retraining the model on a new dataset that consists of prompts followed by the desired output (highly effective for interactive applications like virtual assistants and chatbots)

Question 7

Reinforcement learning from human feedback (RLHF)

Answer 7

• fine-tuning technique
• model is initially trained using supervised learning to predict human-like responses. Then, it is further refined through a reinforcement learning process, where a reward model built from human feedback guides the model toward generating more preferable outputs. Good for sensitive applications.

Question 8

Adapting models for specific domains

Answer 8

This approach involves fine-tuning the model on a corpus of text or data that is specific to a particular industry or sector.

Question 9

Transfer learning

Answer 9

This approach is a method where a model developed for one task is reused as the starting point for a model on a second task (highly efficient in using learned features and knowledge from the general training phase and applying them to a narrower scope with less additional training required.)

Question 10

Continuous pretraining

Answer 10

Pre-training model by continuously feeding it new and emerging data.

Question 11

Key steps in fine-tuning

Answer 11

1) Data curation (this involves a more rigorous selection process to ensure every piece of data is highly relevant.) 2) Labeling 3) Governance and compliance 4) Representativeness and bias checking 5) Feedback integration

Question 12

ROUGE

Answer 12

ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is a set of metrics used to evaluate automatic summarization of texts, in addition to machine translation quality in NLP. The main idea behind ROUGE is to count the number of overlapping units between the model and human text

Question 13

ROUGE-L

Answer 13

This metric uses the longest common subsequence between the generated text and the reference texts. It is particularly good at evaluating the coherence and order of the narrative in the output

Question 14

ROUGE-N

Answer 14

This metric measures the overlap of n-grams between the generated text and the reference text. For example, ROUGE-1 refers to the overlap of unigrams, ROUGE-2 refers to bigrams, and so on. This metric primarily assesses the fluency of the text and the extent to which it includes key ideas from the reference.

Question 15

BLEU

Answer 15

a precision metric used to evaluate the quality of text that has been machine-translated from one natural language to another. BLEU measures the precision of N-grams in the machine-generated text that appears in the reference texts and applies a penalty for overly short translations (brevity penalty).

Question 16

The BERTScore

Answer 16

BERTScore evaluates the semantic similarity rather than relying on exact lexical matches, it is capable of capturing meaning in a more nuanced manner. BERTScore uses the pretrained contextual embeddings from models like BERT to evaluate the quality of text-generation tasks. BERTScore computes the cosine similarity between the contextual embeddings of words in the candidate and the reference texts

Question 17

Hyperparameter tuning

Answer 17

A method to adjust the behavior of an ML algorithm. You can make changes to an ML model by using hyperparameter tuning to modify the behavior of the algorithm.

Question 18

Benchmark dataset

Answer 18

Curated collection of data designed specifically to evaluate the performance of a language model. Can be low cost way to evaluate bias

Question 19

Amazon RAG Vector databases

Answer 19

1) Open Search Service - scalable index management & fastest KNN (nearest neighbor search capabilities)
2) Amazon DocumentDB - No sequal. Millions of vectors
3) Amazon Aurora (relational database)
4) Amazon TDS (relational database)
5) Amazon Neptune (graph database)

Question 20

Model improvement technique costs - from least to most

Answer 20

1) Prompt Engineering (very cheap cause no additional computing or fine tuning)
2) Retrieval Augmented Generation RAG - (cheap bc don’t have to re-computer or fine tune but does require a vector database)
3) Instruction based fine tuning - (fine tuned with specific instructions - required additional computations)
4) Domain adoption fine-tuning (model trained on domain-specific, unlabeled data - intensive computation required).

Question 21

What’s main driver of cost in Amazon Bedrock?

Answer 21

Number of input and output tokens
Side note: batch costs can provide up to 50% savings

Question 22

What’s main driver of cost in Amazon Bedrock?

Answer 22

Number of input and output tokens
Side note: batch costs can provide up to 50% savings

Question 23

Shapely value

Answer 23

Provides a local explanation by quantifying the contribution of each feature to the predictions

Question 24

Partial Dependence Plots (PDP)

Answer 24

Provides global view of models behavior by illustrating how the predicted outcome changes as a single feature is varied

Question 25

Partial Dependence Plots (PDP)

Answer 25

Provides global view of models behavior by illustrating how the predicted outcome changes as a single feature is varied