Domain 3: Application of Foundation Models

Question 1

What’s the goal with Gen AI?

Answer 1

To architect a scalable solution that is efficient and does not reduce the performance of the model.

Question 2

Cost, latency constraints, and required modalities

Answer 2

considerations for apps that use FMs

Question 3

Understanding the requirements for your use case in terms of _____ is important when deciding on AI model.

Answer 3

inference speed

Question 4

Cost

Answer 4

find the balance between training time, cost, and model performance

Question 5

Latency

Answer 5

Consider real-time results requirements, inference times

Question 6

_____ or the duration it takes a model to process data and produce a prediction is another consideration.

Answer 6

Inference speed

Question 7

Modalaties

Answer 7

specific embedding, multi-model, multilingual, pre-trained (architecture/complexity)

Question 8

Accuracy, precision, recall, F1 score, root mean squared error or RMSE, mean average precision or MAP, and mean absolute error, MAE.

Answer 8

standard metrics to evaluate and compare different models

Question 9

It’s important when you are choosing an appropriate metric or set of metrics to _____ before selecting a model.

Answer 9

assess your model’s performance

Question 10

Framework, language, environment, license, documentation, whether the model has been updated and maintained regularly, known issues or limitations, customization, and explainability.

Answer 10

Compatibilities should you consider when using a pre-trained model online

Question 11

What should you look for when considering a pre-trained model?

Answer 11

flexible, modular, transparent, provide tools or methods to visualize or interpret their inner workings, interpret and explain model outcomes

Question 12

T/F: Foundation models are not interpretable by design because they are extremely complex.

Answer 12

True

Question 13

_____ attempts to explain the black box nature of FMs, by approximating it locally with a simpler model that is interpretable.

Answer 13

Explainability

Question 14

T/F: If interpretability is a requirement, then pre-trained foundation models might not be the best choice.

Answer 14

True

Question 15

T/F: Linear regression and decision trees might be better when it comes to explainability.

Answer 15

True

Question 16

The complexity of a model is important and can help you uncover intricate patterns within the data, but it can add challenges to _____ and _____.

Answer 16

maintenance and interpretability

Question 17

T/F: Greater complexity might lead to enhanced performance, but can increase costs.

Answer 17

True

Question 18

T/F: The more complicated the model is, the harder it is to explain the outputs of the model.

Answer 18

True

Question 19

The _____ is where you process new data through the model to make predictions. It is the process of generating an output from an input that you provided to model.

Answer 19

inference

Question 20

_____ gives you the ability to run inference in the foundation model you choose.

Answer 20

Amazon Bedrock

Question 21

A _____, which is an input, is provided to the model for it to generate a response.

Answer 21

prompt

Question 22

_____ are a set of values that can be adjusted to limit or influence the model response.

Answer 22

inference parameters

Question 23

What kind of models can you run inference with?

Answer 23

base, custom, and provision to test FM responses

Question 24

Amazon Bedrock foundation models support the inference parameters of _____ to control randomness and diversity in the response.

Answer 24

temperature, Top K, Top P

Question 25

What parameters are supported by Bedrock to limit the length of responses

Answer 25

response length, penalties, and stop sequences

Question 26

These inputs guide LLMS to generate an appropriate response or output for a given task or instruction.

Answer 26

Prompts

Question 27

You can integrate additional domain-specific data from these data stores or vector data stores that add to your prompts semantically relevant inputs.

Answer 27

retrieval augmented generation, RAG

Question 28

A _____ is a collection of data that is stored as mathematical representations.

Answer 28

vector database

Question 29

It requires millions of graphic processing units, GPUs, compute hours, terabytes and petabytes of data, trillions of tokens, trial and error, and time; generative AI models learn its capabilities.

Answer 29

pre-training

Question 30

_____ add additional capabilities for efficient and fast lookup, and to provide data management, fault tolerance, authentication, and access control and query engine.

Answer 30

Vector databases

Question 31

_____ enhances language models to retrieve and use external knowledge during the generation process. It is a technique in which the retrieval of information from data sources augments the generation of model responses.

Answer 31

RAG

Question 32

RAG combines two components, a _____ component, which searches through a knowledge base and a _____ component, which produces outputs based on the retrieved information.

Answer 32

retriever / generator

Question 33

Why does RAG combine two components?

Answer 33

helps the model access up-to-date and domain-specific knowledge beyond their training data

Question 34

Prompt is passed into the query encoder, which encodes or embeds the data into the same format as the external data. Then the embedding can be passed to the vector database to search and return similar embeddings that have been through the model. Those embeddings are then attached to my new query and can also be mapped back to their original location. If the vector database finds similar data, then the retriever retrieves that data, the LLM combines or augments the new data or text with the original prompt, then the prompt is sent to the LLM to return a completion.

Answer 34

How to use a vector database in the real world

Question 35

How does RAG solve hallucinations?

Answer 35

By complimenting generative LLMs with an external knowledge base that is typically built using a vector database, hydrated with vector-coded knowledge articles

Question 36

Amazon OpenSearch Service, Amazon Aurora, Redis, Amazon Neptune, Amazon DocumentDB with MongoDB compatibility, and Amazon RDS with PostgreSQL

Answer 36

AWS services that help store embeddings within vector databases

Question 37

The _____ delivers low-latency search and aggregations, dashboards, visualization, and dashboarding tools. It also has plugins that provide advanced capabilities such as alerting, fine-grained access control, observability, security monitoring and vector storage and processing. With this service’s vector database capabilities, you can implement semantic search, retrieval of augmented generation, RAG with LLMs, recommendation engines, and search media too.

Answer 37

OpenSearch search engine

Question 38

With _____ you can securely connect foundation models, FMs, to your company data. It is stored as embeddings in the vector engine for more relevant, context-specific, and accurate responses without continuously re-training the FM. Amazon RDS for PostgreSQL also supports the pgvector extension to store embeddings and perform efficient searches.

Answer 38

a fully managed RAG offered by knowledge bases for Amazon Bedrock

Question 39

1. A fully managed AI capability from AWS to help you build applications foundation models.
2. Can automatically break down tasks and generate the required orchestration logic or write custom code, and they can securely connect to your databases through APIs.
3. They can ingest and structure the data for machine consumption and augment it with contextual details to produce more accurate responses and fulfill requests.
4. They are an additional piece of software that orchestrates the prompt completion workflows and interactions between the user requests, foundation model, and external data sources or applications.
5. They automatically call APIs to take actions and invoke knowledge bases to supplement information for these actions.

Answer 39

Agents for Amazon Bedrock

Question 40

_____ are a specific set of inputs provided by you the user. They guide LLMs to generate an appropriate response or output for a given task or instruction.

Answer 40

Prompts

Question 41

A _____ contains components that you want the LLM to perform such as the task or instruction. You might also need the context of that task or instruction and the input text that you want for the response or output.

Answer 41

prompt

Question 42

When you provide a few examples to help the LLM models better perform and calibrate their output to meet your expectations.

Answer 42

few-shot prompting

Question 43

A sentiment classification prompt with no examples provided to the prompt.

Answer 43

zero-shot prompting

Question 44

Where the actual prompt text is replaced with a continuous embedding backer that is optimized during training. This technique helps the prompt to be fine-tuned for a specific task. At the same time, it keeps the rest of the model parameters frozen, which can be more efficient than full fine-tuning.

Answer 44

prompt tuning

Question 45

The practice of crafting and optimizing input prompts. It selects appropriate words, phrases, sentences, punctuation, and separator characters to effectively use LLMs for a wide variety of applications.

Answer 45

prompt engineering

Question 46

Classification, question and answer with and without context, summarization, open-ended text generation, code generation, math, and reasoning or logical thinking.

Answer 46

common tasks supported by LLMs on Amazon Bedrock

Question 47

_____ is the encoded knowledge of language in a large language model. It’s the stored patterns of data that capture relationships and, when prompted, reconstruct language from those patterns. It’s an understanding of patterns that the model can use to generate new outputs, and it’s a statistical database.

Answer 47

Latent space

Question 48

When you write a prompt for a language model, that prompt is ingested by the model and _____. It returns a pile of statistics that then get assembled as words.

Answer 48

refers to its latent space against its database of statistics

Question 49

Designing and refining the input prompts that are fed into the model to guide it towards producing the desired outputs.

Answer 49

prompt engineering

Question 50

First, be specific and provide clear instructions or specifications for the task at hand. For example, include the desired format, examples, comparison, style, tone, output length, and detailed context.

Second, include examples of the desired behavior and direction, such as sample texts, data formats, templates, code, graphs, charts, and more.

Third, experiment and use an iterative process to test prompts and understand how the modifications alter the responses.

Fourth, know the strengths and weaknesses of your model.

Fifth, balance simplicity and complexity in your prompts to avoid vague, unrelated, or unexpected answers.

Six, specifically for your prompt engineers, use multiple comments to offer more context without cluttering your prompt.

Seventh, add guardrails.

Answer 50

prompt engineering techniques

Question 51

Attacks of prompt manipulation with an untrusted input that is created by a user to produce malicious, undesired, or elicit response.

Answer 51

prompt injection

Question 52

When an attacker tries to bypass the guardrails that you have established, this is called _____.

Answer 52

jailbreaking

Question 53

_____ is an attempt to change or manipulate the original prompt with new instructions.

Answer 53

Hijacking

Question 54

_____ is another risk of prompt engineering where harmful instructions are embedded in messages, emails, web pages, and more.

Answer 54

Poisoning

Question 55

1. Use these services to build applications that generate high-quality text for use cases such as content creation summarization, question answering and chatbots.
2. Offer pre-trained language models that can be customized and controlled through prompt engineering.
3. They provide APIs and tools for constructing and refining prompts, along with monitoring and analyzing the resulting outputs.

Answer 55

Amazon Bedrock and Amazon Titan

Question 56

What are the key elements of training a foundation model?

Answer 56

They include pre-training, fine-tuning, and continuous pre-training.

Question 58

With _____, you train the LLM by using huge amounts of unstructured data with self-supervised learning.

Answer 58

pre-training

Question 59

_____ is a process that extends the training of the model to improve the generation of completions for a specific task. It is a supervised learning process and you use a dataset of labeled examples to update the weights of the LLM, it helps to adapt foundation models to your custom datasets and use cases.

Answer 59

Fine-tuning

Question 60

_____ happens when the whole fine-tuning process modifies the weights of the original LLM. This can improve the performance of the single task fine-tuning, but it can degrade performance on other tasks.

Answer 60

Catastrophic forgetting

Question 61

When you train and tune a foundation model, _____.

Answer 61

you load the model parameters and add memory for the optimizer, gradients, forward activations, and temporal memory

Question 62

_____ is a process and set of techniques that freeze or preserve the parameters and weights of the original LLM and fine-tune or train a small number of task-specific adaptor layers and parameters. It reduces the compute and memory that’s needed because it’s fine-tuning a small set of model parameters.

Answer 62

Parameter-efficient fine-tuning, PEFT

Question 63

_____ is a popular PEFT technique that also preserves or freezes the original weights of the foundation model and creates new trainable low-rank matrices into each layer of a transformer architecture.

Answer 63

Low-rank adaptation or LoRA

Question 64

PEFT and LoRA modify the _____ of your model, but not the representations.

Answer 64

weights

Question 65

_____ encode semantic information similar to embeddings.

Answer 65

Representations

Question 66

_____ is a fine-tuning process that freezes the base model and learns task-specific interventions on hidden representations.

Answer 66

Representation fine-tuning, ReFT

Question 67

The _____ says that concepts are encoded in linear subspaces of representation in a neural network.

Answer 67

linear representation hypothesis

Question 68

_____ is an extension of fine-tuning a single task. This requires a lot of data. For this process, the training dataset has examples of inputs and outputs for multiple tasks.

Answer 68

Multitask fine-tuning

Question 69

_____ gives you the ability to use the pre-trained foundation models and adapt them to specific tasks by using limited domain-specific data. You can use this to help your model work with domain-specific language such as industry jargon, technical terms, or other specialized data.

Answer 69

Domain adaptation fine-tuning

Question 70

_____ provides the capability to fine-tune a large language model, particularly a text generation model on the domain- specific dataset so you can improve the performance of your model and help it better understand human-like prompts to generate human-like responses.

Answer 70

Amazon SageMaker JumpStart

Question 71

During _____, you select prompts from your training dataset and pass them to the LLM to generate completions. Then, compare the distribution of completions, and the training label, to calculate a loss between the two token distributions, which you can use to update your model’s weights so the model’s performance on the task improves.

Answer 71

fine-tuning

Question 72

You can define separate evaluation steps to measure your LLM’s performance, by using the _____. You will get the validation accuracy, and after you’ve completed your fine-tuning, you can perform a final performance evaluation by using this, and the last result will give you the test accuracy.

Answer 72

holdout validation dataset

Question 73

In machine learning, _____ is the collecting pre-processing, and organizing of your raw data. for your model.

Answer 73

data preparation

Question 74

If you have low-code data preparation, you can use _____ to create data flows that define your ML data pre-processing. These data flows feature engineering workflows that use little to no coding.

Answer 74

Amazon SageMaker Canvas

Question 75

If you have _____ that needs to scale, you can use open source frameworks such as Apache Spark, Apache Hive, or Presto.

Answer 75

data preparation

Question 76

If you need to use structured query language, or SQL, in SageMaker Studio for data preparation, you can use _____.

Answer 76

Jupyter Lab

Question 77

If you have data preparation for feature discovery and storage, you can use _____ to search, discover, and retrieve features for model training. You can also use it to provide a centralized repository to store feature data in a standardized format.

Answer 77

Amazon SageMaker Feature Store

Question 78

You can use _____ to analyze your data and detect potential biases across multiple facets, which can help you detect whether your training data contains imbalanced representations or labeling biases between groups such as gender, race, or age.

Answer 78

Amazon SageMaker Clarify

Question 79

If you have data that needs to be labeled, you can use _____ to manage the data labeling workflows for your training datasets.

Answer 79

SageMaker Ground Truth

Question 80

T/F: The output of generative AI models is non-deterministic, which makes validation more difficult.

Answer 80

True

Question 81

1. Improve application performance by reducing the size of the LLMs. This action can reduce the inference latency because the smaller size model loads more quickly. However, remember that reducing the size of the model might decrease its performance.
2. Make a more concise prompt, reducing the size of the retrieved snippets and their number, and reducing generation through inference parameters and prompt.

Answer 81

optimization techniques

Question 82

_____ is a set of metrics and a software package. It is used to evaluate automatic summarization tasks and machine translation software in natural language processing. It evaluates how well the input compares to the generated output.

Answer 82

Recall Oriented Understudy for Gisting Evaluation, or ROUGE

Question 83

_____ is an algorithm that is used for translation tasks. It evaluates the quality of text which has been machine translated from one natural language to another.

Answer 83

Bilingual Evaluation Understudy, or BLEU

Question 84

_____ was created to help the development of models that can generalize across multiple tasks. It is a collection of natural language tasks, such as sentiment analysis and question answering. You can use these tasks to evaluate and compare model performance across a set of language tasks. Then, you can use the benchmark to measure and compare the model performance.

Answer 84

GLUE

Question 85

_____ was introduced in 2019 and adds additional tasks, such as multi-sentence reasoning and reading comprehension.

Answer 85

SuperGlue

Question 86

_____ evaluates the knowledge and problem-solving capabilities of the model.

Answer 86

Massive Multitask Language Understanding, MMLU

Question 87

____ focuses on tasks that are beyond the capabilities of the current language models. It contains tasks such as math, biology, physics, bias, linguistics, reasoning, childhood development, software development, and more.

Answer 87

The Beyond the Imitation Game Benchmark, BIG-bench

Question 88

Another benchmark is the _____ which is a benchmark to help improve model transparency. It offers users guidance on which model performs well for a given task. This is a combination of metrics for tasks such as summarization, question and answer, sentiment analysis, and bias detection.

Answer 88

Holistic Evaluation of Language Models, HELM

Question 89

You can also use _____ to manually evaluate your model responses. For example, you can use these to compare the responses of SageMaker JumpStart models, and you can also specify responses from models outside AWS.

Answer 89

human workers

Question 90

You can use _____ to evaluate large language models, LLMs, and create model evaluation jobs. A model evaluation job helps to evaluate and compare model quality and metrics for text-based foundation models from SageMaker JumpStart.

Answer 90

Amazon SageMaker Clarify

Question 91

_____ provides an evaluation module that can automatically compare generated responses and calculate a semantic similarity base score, BERTscore, against a human reference. It is suitable to evaluate faithfulness and hallucinations in text-generation tasks

Answer 91

Amazon Bedrock

Question 92

_____ helps with the challenge of internal knowledge of models being outdated. If your model is outdated, then it’ll not know newer information. This helps by providing a context, which helps to avoid hallucinations and improve factuality by grounding responses.

Answer 92

RAG

Question 93

1. Use an orchestration library to configure and manage the passing of user input to the large language model and the return of completions.
2. RAG helps your model access additional external data at inference times. And the additional external data can help improve the relevance and accuracy of completions.
3. RAG helps overcome the outdated knowledge issue if your model uses older information.

Answer 93

Actions to give you more configurations to connect your LLM to external components and integrate deployment within your application

Question 94

1. define business goals
2. determine metrics then measure, monitor, review them
3. make sure models interact well w/ your other systems (have to interact in real time using APIs and interfaces)
4. choose the large language models to use with your application and the appropriate infrastructure for your inference needs (remember storage)
5. consider additional tools/frameworks for LLMs
6. user interface (like website/rest API) to consume app

Answer 94

Primary AI considerations

Question 95

The _____provides the compute, storage, and network to serve and host your LLMs and to host your application components. For this layer, you must ensure that your data is being handled securely across the AI lifecycle for data preparation, training, and inferencing.

Answer 95

infrastructure layer