Responsible Artificial Intelligence Practices

Question 1

Algorithm Bias

Answer 1

The algorithms and models used in AI systems can introduce biases, even if the training data is unbiased. This can happen due to the inherent assumptions or simplifications made by the algorithms, in particular for underrepresented groups, or due to machine learning models optimize for performance, not necessarily for fairness.

Question 2

Interaction Bias

Answer 2

Biases can also arise from the way humans interact with AI systems or the context in which the AI is deployed. For example, if an AI system for facial recognition is primarily tested on a certain demographic group, it may perform poorly on other groups.

Question 3

Bias amplification

Answer 3

AI systems can amplify and perpetuate existing societal biases, if not properly designed and monitored. This can lead to unfair treatment or discrimination against certain groups, even if it was not intentional. And with more adoption of AI, there is increased risk of bias amplifying further, especially through social media platforms.

Question 4

Toxicity

Answer 4

Toxicity is the possibility of generating content (whether it be text, images, or other modalities) that is offensive, disturbing, or otherwise inappropriate. This is a primary concern with generative AI. It is hard to even define and scope toxicity.

Question 5

Intellectual Propriety

Answer 5

Protecting intellectual property was a problem with early LLMs. This was because the LLMs had a tendency to occasionally produce text or code passages that were verbatim of parts of their training data, resulting in privacy and other concerns. But even improvements in this regard have not prevented reproductions of training content that are more ambiguous and nuanced.

Question 6

Transparency

Answer 6

The transparency dimension of responsible AI refers to the practice of how you might communicate information about an AI system. This helps stakeholders to make informed choices about their use of the system. Some of this information includes development processes, system capabilities, and limitations.

Question 7

Veracity and robustness in AI

Answer 7

refers to the mechanisms to ensure an AI system operates reliably, even with unexpected situations, uncertainty, and errors.

Question 8

Safety in responsible AI

Answer 8

refers to the development of algorithms, models, and systems in such a way that they are responsible, safe, and beneficial for individuals and society as a whole.

Question 9

Model evaluation on Amazon Bedrock,

Answer 9

you can evaluate, compare, and select the best foundation model for your use case in just a few clicks. Amazon Bedrock offers a choice of automatic evaluation and human evaluation.

Question 10

Model evaluation on Amazon Bedrock: Automatic evaluation

Answer 10

offers predefined metrics such as accuracy, robustness, and toxicity.

Question 11

Model evaluation on Amazon Bedrock: Human evaluation offers

Answer 11

subjective or custom metrics such as friendliness, style, and alignment to brand voice. For human evaluation, you can use your in-house employees or an AWS-managed team as reviewers.

Question 12

SageMaker Clarify supports FM evaluation.

Answer 12

You can automatically evaluate FMs for your generative AI use case with metrics such as accuracy, robustness, and toxicity to support your responsible AI initiative.

Question 13

Guardrails for Amazon Bedrock gives you the ability to

Answer 13

configure thresholds across the different categories to filter out harmful interactions.

Question 14

Guardrails for Amazon Bedrock helps you detect PII in

Answer 14

user inputs and FM responses. Based on the use case, you can selectively reject inputs containing PII or redact PII in FM responses

Question 15

SageMaker Clarify helps identify potential

Answer 15

bias in machine learning models and datasets without the need for extensive coding. You specify input features, such as gender or age, and SageMaker Clarify runs an analysis job to detect potential bias in those features.

Question 16

SageMaker Data Wrangler offers three balancing operators:

Answer 16

random undersampling, random oversampling, and Synthetic Minority Oversampling Technique (SMOTE) to rebalance data in your unbalanced datasets.

Question 17

Amazon SageMaker Data Wrangler to

Answer 17

balance your data in cases of any imbalances.

Question 18

SageMaker Experiments is a capability of

Answer 18

SageMaker that you can use to create, manage, analyze, and compare your machine learning experiments.

Question 19

SageMaker Clarify is integrated with Amazon SageMaker Experiments to provide scores detailing which features

Answer 19

contributed the most to your model prediction on a particular input for tabular, natural language processing (NLP), and computer vision models.

Question 20

Amazon SageMaker Model Monitor monitors the

Answer 20

quality of your models.
You can set up continuous monitoring with a real-time endpoint (or a batch transform job that runs regularly), or on-schedule monitoring for asynchronous batch transform jobs. With SageMaker Model Monitor, you can set alerts that notify you when there are deviations in the model quality. With early and proactive detection of these deviations, you can take corrective actions.

Question 21

Amazon Augmented AI (Amazon A2I) is a service that helps build the workflows required for human review of ML predictions.

Answer 21

Amazon A2I brings human review to all developers and removes the undifferentiated heavy lifting associated with building human review systems or managing large numbers of human reviewers.

Question 22

Amazon SageMaker Model Cards:

Answer 22

With SageMaker Model Cards, you can capture, retrieve, and share essential model information, such as intended uses, risk ratings, and training details, from conception to deployment.

Question 23

With SageMaker Model Cards,

Answer 23

you can capture, retrieve, and share essential model information, such as intended uses, risk ratings, and training details, from conception to deployment.

Question 24

AWS AI Service Cards are a new resource to help you better understand AWS AI services.

Answer 24

AI Service Cards are a form of responsible AI documentation that provides a single place to find information on the intended use cases and limitations, responsible AI design choices, and deployment and performance optimization best practices for AWS AI services.

Question 25

When selecting a model for your AI application, you must

Answer 25

narrowly define your use case. This is important because you can tune your model for that specific use case.

Question 26

Responsible agency in responsible AI refers to

Answer 26

an AI system’s capacity to make good judgments and act in a socially responsible manner.

Question 27

Data augmentation can be used to

Answer 27

generate new instances of underrepresented groups. This can help to balance the dataset and prevent biases towards more represented groups.

Question 28

SageMaker Model Dashboard can be used as

Answer 28

a central place to keep the team informed on model behavior in production.

Question 29

Transparency helps to understand

Answer 29

HOW a model makes decisions.

This helps to provide accountability and builds trust in the AI system. Transparency also makes auditing a system easier.

Question 30

Explainability helps to understand

Answer 30

WHY the model made the decision that it made. It gives insight into the limitations of a model.

This helps developers with debugging and troubleshooting the model. It also allows users to make informed decisions on how to use the model.

Question 31

There are several explainability frameworks available that can help summarize and interpret the decisions made by AI systems. Name 3

Answer 31

, such as SHapley Value Added (SHAP),

Local Interpretable Model-Agnostic Explanations (LIME), and Counterfactual Explanations,

Question 32

To help with transparency, Amazon offers

Answer 32

AWS AI Service Cards and Amazon SageMaker Model Cards. The difference between them is that with AI Service Cards, Amazon provides transparent documentation on Amazon services that help you build your AI services. With SageMaker Model Cards, you can catalog and provide documentation on models that you create or develop yourself.

Question 33

The difference between AWS AI Service Cards and Amazon SageMaker Model Cards. T

Answer 33

he difference between them is that with AI Service Cards, Amazon provides transparent documentation on Amazon services that help you build your AI services. With SageMaker Model Cards, you can catalog and provide documentation on models that you create or develop yourself.

Question 34

SageMaker Model Cards

Answer 34

document critical details about your ML models in a single place for streamlined governance and reporting.

Catalog details include information such as the intended use and risk rating of a model, training details and metrics, evaluation results and observations, and additional callouts such as considerations, recommendations, and custom informatio

Question 35

Two AWS tools for explainability

Answer 35

SageMaker Clarify
Amazon SageMaker Autopilot uses tools provided by SageMaker Clarify

Question 36

SageMaker Autopilot explanatory functionality determines the contribution of individual features or inputs to the model’s output and provides insights into the relevance of different features. You can use it to understand why a model made a prediction after training or use it to provide per-instance explanation during inference.

Question 37

Bias-variance tradeoff is when you optimize your model with the right balance between bias and variance. This means that you need to optimize your model so that it is not underfitted or overfitted. The goal is to achieve a trained model with the lowest bias and lowest variance tradeoff for a given data set.

Question 38

Dimension reduction is

Answer 38

an unsupervised machine learning algorithm that attempts to reduce the dimensionality (number of features) within a dataset while still retaining as much information as possible.

Question 39

offers the most comprehensive set of human-in-the-loop capabilities for incorporating human feedback across the ML lifecycle to improve model accuracy and relevancy

Answer 39

SageMaker Ground Truth

Question 40

Design for amplified decision-making is

Answer 40

a design principle that supports decision-makers in making decisions carefully in high-pressure environments by using technology

Question 41

To avoid underfitting and overfitting the data, the model should be trained with low bias and low variance.

Answer 41

If either the bias or the variance is high, the model will not perform well.