AI Practice Test #5

Question 1

Amazon Q Developer

Answer 1

Amazon Q Developer assists developers and IT professionals with all their tasks—from coding, testing, and upgrading applications, to diagnosing errors, performing security scanning and fixes, and optimizing AWS resources.

Question 2

Amazon Q Developer (capabilities)

Answer 2

(1) Understand and manage your cloud infrastructure on AWS

Amazon Q Developer helps you understand and manage your cloud infrastructure on AWS. With this capability, you can list and describe your AWS resources using natural language prompts, minimizing friction in navigating the AWS Management Console and compiling all information from documentation pages.

For example, you can ask Amazon Q Developer, “List all of my Lambda functions”. Then, Amazon Q Developer returns the response with a set of my AWS Lambda functions as requested, as well as deep links so you can navigate to each resource easily.

(2) Get answers to your AWS account-specific cost-related questions using natural language

Amazon Q Developer can get answers to AWS cost-related questions using natural language. This capability works by retrieving and analyzing cost data from AWS Cost Explorer.

Question 3

Amazon Q Developer (team’s development efforts)

Answer 3

Amazon Q Developer can suggest code snippets, providing developers with recommendations for code based on specific tasks or requirements

This is the correct option because Amazon Q Developer is designed to assist developers by providing code suggestions and recommendations that align with their coding tasks. It leverages machine learning models trained on vast datasets to suggest code snippets, optimize code efficiency, and help developers follow best practices. This functionality helps speed up development processes and enhances productivity.

Amazon Q Developer is not specifically designed to handle the full deployment process of applications.

Question 4

Self-supervised learning

Answer 4

It works when models are provided vast amounts of raw, almost entirely, or completely unlabeled data and then generate the labels themselves.

Foundation models use self-supervised learning to create labels from input data. In self-supervised learning, models are provided vast amounts of raw completely unlabeled data and then the models generate the labels themselves. This means no one has instructed or trained the model with labeled training data sets.

Question 5

Supervised learning

Answer 5

In supervised learning, models are supplied with labeled and defined training data to assess for correlations. The sample data specifies both the input and the output for the model. For example, images of handwritten figures are annotated to indicate which number they correspond to. A supervised learning system could recognize the clusters of pixels and shapes associated with each number, given sufficient examples.

Question 6

Unsupervised learning

Answer 6

Unsupervised learning algorithms train on unlabeled data. They scan through new data, trying to establish meaningful connections between the inputs and predetermined outputs. They can spot patterns and categorize data. For example, unsupervised algorithms could group news articles from different news sites into common categories like sports, crime, etc. They can use natural language processing to comprehend meaning and emotion in the article.

Question 7

Amazon SageMaker Model Cards

Answer 7

Describes how a model should be used in a production environment

Use Amazon SageMaker Model Cards to document critical details about your machine learning (ML) models in a single place for streamlined governance and reporting.

Catalog details such as the intended use and risk rating of a model, training details and metrics, evaluation results and observations, and additional call-outs such as considerations, recommendations, and custom information.

Model cards provide prescriptive guidance on what information to document and include fields for custom information. Specifying the intended uses of a model helps ensure that model developers and users have the information they need to train or deploy the model responsibly.

The intended uses of a model go beyond technical details and describe how a model should be used in production, the scenarios in which is appropriate to use a model, and additional considerations such as the type of data to use with the model or any assumptions made during development.

Question 8

Machine Learning models

Answer 8

Machine Learning models can be deterministic or probabilistic or a mix of both

Machine Learning models can be deterministic or probabilistic or a mix of both, depending on their nature and how they are designed to operate.

Deterministic models always produce the same output given the same input. Their behavior is predictable and consistent. Example: Decision Trees: Given the same input data, a decision tree will always follow the same path and produce the same output.

Probabilistic models provide a distribution of possible outcomes rather than a single output. They incorporate uncertainty and randomness in their predictions. Example: Bayesian Networks: These models represent probabilistic relationships among variables and provide probabilities for different outcomes.

Some models combine both deterministic and probabilistic elements, such as neural networks and random forests.

Question 9

Dynamic prompt engineering

Answer 9

Implement dynamic prompt engineering to customize responses based on user characteristics like age

Dynamic prompt engineering involves modifying the input prompts to the Large Language Model (LLM) to customize the chatbot’s responses based on the user’s age. By altering the prompt dynamically, you can provide specific instructions or context to the LLM to generate age-appropriate responses. For example, if the user is a child, the prompt might include instructions to use simpler language or a friendly tone. This approach does not require changing the model itself and leverages Amazon Bedrock’s ability to interpret context from customized prompts effectively.

To provide custom responses via an LLM chatbot built using Amazon Bedrock based on the user’s age, you can implement a strategy that dynamically adjusts the chatbot’s responses according to the age group of the user. For the given use case, you can leverage Amazon Bedrock to build a custom prompt logic for the LLM that dynamically adjusts the input prompt based on the user’s age category, like the following example in Python:

Then, use the Amazon Bedrock API to send the customized prompts to the foundation model. The Bedrock service will generate responses based on the context provided in each prompt, adapting the output to fit the desired style and tone for the specific age group.

Question 10

Retrieval-Augmented Generation (RAG)

Answer 10

RAG is a technique that combines a retrieval mechanism (which fetches relevant documents or data from a knowledge base) with a generation model to provide more factual and context-rich responses. While RAG can enhance response accuracy by adding external context, it is not specifically designed for customizing responses based on user characteristics like age. RAG focuses on improving the relevance and factual accuracy of outputs, not on adapting the style or complexity of the language to suit different age groups.

Question 11

model re-training

Answer 11

Re-training the model involves using a large dataset to update the entire model’s parameters, which is time-consuming, costly, and unnecessary for simply tailoring responses based on user age. Amazon Bedrock provides access to pre-trained foundation models that are already capable of generating diverse outputs based on the input prompts. Re-training is overkill for this task and is not the appropriate solution for generating age-specific responses dynamically.

Question 12

Fine-tuning

Answer 12

Fine-tuning involves training the LLM on a specialized dataset to improve its performance on specific tasks or domains. However, this method is more suited for developing domain-specific expertise in the model rather than adjusting the style or tone of responses based on user age. Fine-tuning can be resource-intensive and time-consuming, and it is not necessary for generating age-appropriate responses when prompt engineering can dynamically handle the customization without modifying the model itself.

Question 13

AWS Audit Manager

Answer 13

AWS Audit Manager helps automate the collection of evidence to continuously audit your AWS usage. It simplifies the process of assessing risk and compliance with regulations and industry standards, making it an essential tool for governance in AI systems.

Question 14

AWS Artifact

Answer 14

AWS Artifact provides on-demand access to AWS’ compliance reports and online agreements. It is useful for obtaining compliance documentation but does not provide continuous auditing or automated evidence collection.

Question 15

AWS Trusted Advisor

Answer 15

AWS Trusted Advisor offers guidance to help optimize your AWS environment for cost savings, performance, security, and fault tolerance. While it provides recommendations for best practices, it does not focus on auditing or evidence collection for compliance.

Question 16

AWS CloudTrail

Answer 16

AWS CloudTrail records AWS API calls for auditing purposes and delivers log files for compliance and operational troubleshooting. It is crucial for tracking user activity but does not automate compliance assessments or evidence collection.

Question 17

Amazon Q in Connect

Answer 17

Amazon Connect is the contact center service from AWS. Amazon Q helps customer service agents provide better customer service. Amazon Q in Connect uses real-time conversation with the customer along with relevant company content to automatically recommend what to say or what actions an agent should take to better assist customers.

Question 18

Amazon Q Business

Answer 18

Amazon Q Business is a fully managed, generative-AI powered assistant that you can configure to answer questions, provide summaries, generate content, and complete tasks based on your enterprise data. It allows end users to receive immediate, permissions-aware responses from enterprise data sources with citations, for use cases such as IT, HR, and benefits help desks.

Question 19

Amazon Q in QuickSight

Answer 19

With Amazon Q in QuickSight, customers get a generative BI assistant that allows business analysts to use natural language to build BI dashboards in minutes and easily create visualizations and complex calculations.

Question 20

Semi-supervised learning

Answer 20

Semi-supervised learning is when you apply both supervised and unsupervised learning techniques to a common problem. This technique relies on using a small amount of labeled data and a large amount of unlabeled data to train systems. First, the labeled data is used to partially train the machine learning algorithm. After that, the partially trained algorithm labels the unlabeled data. This process is called pseudo-labeling. The model is then re-trained on the resulting data mix without being explicitly programmed.

Question 21

Fraud identification

Answer 21

Within a large set of transactional data, there’s a subset of labeled data where experts have confirmed fraudulent transactions. For a more accurate result, the machine learning solution would train first on the unlabeled data and then with the labeled data.

Question 22

Sentiment analysis

Answer 22

When considering the breadth of an organization’s text-based customer interactions, it may not be cost-effective to categorize or label sentiment across all channels. An organization could train a model on the larger unlabeled portion of data first, and then a sample that has been labeled. This would provide the organization with a greater degree of confidence in customer sentiment across the business.

Question 23

Neural network

Answer 23

A neural network solution is a more complex supervised learning technique. To produce a given outcome, it takes some given inputs and performs one or more layers of mathematical transformation based on adjusting data weightings. An example of a neural network technique is predicting a digit from a handwritten image.

Question 24

Clustering

Answer 24

Clustering is an unsupervised learning technique that groups certain data inputs, so they may be categorized as a whole. There are various types of clustering algorithms depending on the input data. An example of clustering is identifying different types of network traffic to predict potential security incidents.

Question 25

Dimensionality reduction

Answer 25

Dimensionality reduction is an unsupervised learning technique that reduces the number of features in a dataset. It’s often used to preprocess data for other machine learning functions and reduce complexity and overheads. For example, it may blur out or crop background features in an image recognition application.

Question 26

Amazon SageMaker Feature Store

Answer 26

Amazon SageMaker Feature Store is a fully managed, purpose-built repository to store, share, and manage features for machine learning (ML) models. Features are inputs to ML models used during training and inference. For example, in an application that recommends a music playlist, features could include song ratings, listening duration, and listener demographics.

You can ingest data into SageMaker Feature Store from a variety of sources, such as application and service logs, clickstreams, sensors, and tabular data from Amazon Simple Storage Service (Amazon S3), Amazon Redshift, AWS Lake Formation, Snowflake, and Databricks Delta Lake.

Question 27

Amazon SageMaker Clarify

Answer 27

SageMaker Clarify helps identify potential bias during data preparation without writing code. You specify input features, such as gender or age, and SageMaker Clarify runs an analysis job to detect potential bias in those features.

Question 28

Amazon SageMaker Data Wrangler

Answer 28

Amazon SageMaker Data Wrangler reduces the time it takes to aggregate and prepare tabular and image data for ML from weeks to minutes. With SageMaker Data Wrangler, you can simplify the process of data preparation and feature engineering, and complete each step of the data preparation workflow (including data selection, cleansing, exploration, visualization, and processing at scale) from a single visual interface.

Question 29

Amazon SageMaker Ground Truth

Answer 29

Amazon SageMaker Ground Truth offers the most comprehensive set of human-in-the-loop capabilities, allowing you to harness the power of human feedback across the ML lifecycle to improve the accuracy and relevancy of models. You can complete a variety of human-in-the-loop tasks with SageMaker Ground Truth, from data generation and annotation to model review, customization, and evaluation, either through a self-service or an AWS-managed offering.

Question 30

Amazon SageMaker Model Monitor

Answer 30

Amazon SageMaker Model Monitor is a service that continuously monitors the quality of machine learning models in production and helps detect data drift, model quality issues, and anomalies. It ensures that models perform as expected and alerts users to issues that might require human intervention.

Amazon SageMaker Model Monitor monitors the quality of Amazon SageMaker machine learning models in production. With Model Monitor, you can set up: Continuous monitoring with a real-time endpoint, Continuous monitoring with a batch transform job that runs regularly, and On-schedule monitoring for asynchronous batch transform jobs.

Question 31

Amazon Augmented AI (Amazon A2I)

Answer 31

Amazon Augmented AI (A2I) is a service that helps implement human review workflows for machine learning predictions. It integrates human judgment into ML workflows, allowing for reviews and corrections of model predictions, which is critical for applications requiring high accuracy and accountability.

Question 32

Amazon Titan

Answer 32

Amazon Titan foundation models, developed by Amazon Web Services (AWS), are pre-trained on extensive datasets, making them robust and versatile models suitable for a wide range of applications. Amazon Titan foundation models (FMs) provide customers with a breadth of high-performing image, multimodal, and text model choices, via a fully managed API. Amazon Titan models are created by AWS and pretrained on large datasets, making them powerful, general-purpose models built to support a variety of use cases, while also supporting the responsible use of AI.

Question 33

Llama

Answer 33

Llama is a series of large language models trained on publicly available data. They are built on the transformer architecture, enabling them to handle input sequences of any length and produce output sequences of varying lengths. A notable feature of Llama models is their capacity to generate coherent and contextually appropriate text.

Question 34

Jurassic

Answer 34

Jurassic family of models from AI21 Labs supported use cases such as question answering, summarization, draft generation, advanced information extraction, and ideation for tasks requiring intricate reasoning and logic.

Question 35

Claude

Answer 35

Claude is Anthropic’s frontier, state-of-the-art large language model that offers important features for enterprises like advanced reasoning, vision analysis, code generation, and multilingual processing.

Question 36

Infrastructure as a Service (IaaS)

Answer 36

Cloud Computing can be broadly divided into three types - Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS).

IaaS contains the basic building blocks for cloud IT. It typically provides access to networking features, computers (virtual or on dedicated hardware), and data storage space. IaaS gives the highest level of flexibility and management control over IT resources.

EC2 gives you full control over managing the underlying OS, virtual network configurations, storage, data, and applications. So EC2 is an example of an IaaS service.

Question 37

Platform as a Service (PaaS)

Answer 37

PaaS removes the need to manage underlying infrastructure (usually hardware and operating systems), and allows you to focus on the deployment and management of your applications. You don’t need to worry about resource procurement, capacity planning, software maintenance, patching, or any of the other undifferentiated heavy lifting involved in running your application.

Elastic Beanstalk is an example of a PaaS service. You can simply upload your code and Elastic Beanstalk automatically handles the deployment, from capacity provisioning, load balancing, and auto-scaling to application health monitoring.

Question 38

Software as a Service (SaaS)

Answer 38

SaaS provides you with a complete product that is run and managed by the service provider. With a SaaS offering, you don’t have to think about how the service is maintained or how the underlying infrastructure is managed. You only need to think about how you will use that particular software. AWS Rekognition is an example of a SaaS service.

Question 39

Amazon SageMaker Model Dashboard

Answer 39

Amazon SageMaker Model Dashboard is a centralized repository of all models created in your account. The models are generally the outputs of SageMaker training jobs, but you can also import models trained elsewhere and host them on SageMaker. Model Dashboard provides a single interface for IT administrators, model risk managers, and business leaders to track all deployed models and aggregate data from multiple AWS services to provide indicators about how your models are performing. You can view details about model endpoints, batch transform jobs, and monitoring jobs for additional insights into model performance.

The dashboard’s visual display helps you quickly identify which models have missing or inactive monitors, so you can ensure all models are periodically checked for data drift, model drift, bias drift, and feature attribution drift. Lastly, the dashboard’s ready access to model details helps you dive deep, so you can access logs, infrastructure-related information, and resources to help you debug monitoring failures.

Question 40

FMs use:

Labeled or unlabeled data?

Supervised or Self-supervised learning?

Answer 40

FMs use unlabeled training data sets for self-supervised learning

Self-supervised learning is a machine learning approach that applies unsupervised learning methods to tasks usually requiring supervised learning. Instead of using labeled datasets for guidance, self-supervised models create implicit labels from unstructured data.

Foundation models use self-supervised learning to create labels from input data. This means no one has instructed or trained the model with labeled training data sets.

Question 41

Accuracy

Answer 41

Accuracy, which measures the proportion of correctly predicted instances (both true positives and true negatives) out of the total number of instances

Accuracy is the most appropriate metric when the goal is to understand the proportion of correct outcomes in a binary classification problem. It provides a straightforward measure of how often the model correctly predicts the positive and negative classes. Accuracy is a suitable choice when the dataset is balanced (i.e., the number of positive and negative instances is approximately equal) and when the company wants a simple, overall performance measure of the model’s correctness.

Question 42

Root Mean Squared Error (RMSE)

Answer 42

Root Mean Squared Error (RMSE), a metric that calculates the square root of the average of the squared differences between predicted and actual values

RMSE is a metric used to measure the average magnitude of errors in a regression model’s predictions.

It is not appropriate for binary classification tasks because it is designed to assess continuous numeric predictions rather than categorical outcomes. Therefore, RMSE does not provide meaningful insights into the correct or incorrect outcomes in a classification context.

Question 43

R-squared

Answer 43

R-squared, a statistical measure that indicates the proportion of variance in the dependent variable explained by the independent variables

R-squared is a metric that measures the goodness of fit in regression models. It shows how well the independent variables explain the variance in the dependent variable. Since R-squared is specific to regression tasks and not applicable to classification problems, it is not a suitable metric for evaluating the correct outcomes in a binary classification scenario.

Question 44

F1 Score

Answer 44

F1 Score, a metric that considers both precision and recall by calculating their harmonic mean

The F1 Score is a useful metric when dealing with imbalanced datasets in binary classification, as it balances precision (the proportion of true positive predictions among all positive predictions) and recall (the proportion of true positives among all actual positive instances). However, if the primary focus is simply to measure the correct outcomes without concern for the balance between precision and recall, then accuracy is a more straightforward metric. F1 Score is most appropriate when both false positives and false negatives need to be minimized, but it may not be necessary if the dataset is balanced and the company only wants to know the overall proportion of correct predictions.

Question 45

Best-fit use cases for utilizing Retrieval Augmented Generation (RAG) in Amazon Bedrock?

Answer 45

Customer service chatbot

Medical queries chatbot

To equip foundation models (FMs) with up-to-date and proprietary information, organizations use Retrieval Augmented Generation (RAG), a technique that fetches data from company data sources and enriches the prompt to provide more relevant and accurate responses. Knowledge Bases for Amazon Bedrock is a fully managed capability that helps you implement the entire RAG workflow from ingestion to retrieval and prompt augmentation without having to build custom integrations to data sources and manage data flows. Some of the common use cases that can be addressed via RAG in Amazon Bedrock are customer service chatbot, medical queries chatbot, legal research and analysis, etc.

It is NOT the right fit for:
(a) Original content creation
(b) Image generation from text prompt
(c) Product recommendations that match shopper preferences

Question 46

The bias versus variance trade-off

(Got this correct)

Answer 46

The bias versus variance trade-off refers to the challenge of balancing the error due to the model’s complexity (variance) and the error due to incorrect assumptions in the model (bias), where high bias can cause underfitting and high variance can cause overfitting

Question 47

High Bias

Answer 47

Underfitting

The high-bias model will not be able to capture the dataset trend. It is considered as the underfitting model which has a high error rate. It is due to a very simplified algorithm.

One of the main reasons for high bias is the very simplified model.

https://www.geeksforgeeks.org/bias-vs-variance-in-machine-learning/

Question 48

High Variance

Answer 48

Overfitting

High variance means that the model is very sensitive to changes in the training data and can result in significant changes in the estimate of the target function when trained on different subsets of data from the same distribution. This is the case of overfitting when the model performs well on the training data but poorly on new, unseen test data. It fits the training data too closely that it fails on the new training dataset.

Variance is the amount by which the performance of a predictive model changes when it is trained on different subsets of the training data.