week 3 Flashcards
What is XAI (Explainable AI)?
XAI focuses on making AI models transparent and understandable, aiding debugging, improvement, and building trust.
What are the two main approaches to achieving model understanding?
- Build inherently explainable models (e.g., decision trees, linear regression). 2. Explain pre-built models in a post-hoc manner (e.g., LIME, SHAP).
What are intrinsic methods in XAI?
Intrinsic methods are explanations built into the model itself, such as decision trees or linear regression models.
What are post-hoc methods in XAI?
Post-hoc methods are explanations applied after the model is built, like LIME or SHAP, and can be model-agnostic.
What is the difference between model-specific and model-agnostic methods?
Model-specific methods are tailored to particular model types, while model-agnostic methods can be applied to any model.
What are some examples of model-specific techniques?
Techniques include ANOVA for statistical analysis, variable importance in random forests, and partial dependence plots.
How does LIME explain model predictions?
LIME creates a simple, interpretable model around the specific instance being explained by perturbing feature values and fitting a local surrogate model.
What is the main goal of LIME’s objective function?
To create a surrogate model that is both faithful to the original complex model and simple enough to be interpretable in the local neighborhood of the instance.
What is SHAP and how does it work?
SHAP (SHapley Additive exPlanations) attributes model predictions to features using game theory principles, providing feature importance and direction of influence.
What are local vs. global explanations?
Local explanations clarify why a specific instance was predicted, while global explanations elucidate the overall model behavior.
Why is it important to evaluate explanations in XAI?
To ensure explanations accurately reflect model reasoning, meet stakeholder needs, and identify potential biases.
What are key evaluation criteria for XAI explanations?
Fidelity, comprehensibility, sufficiency, and trustworthiness are critical criteria for evaluating explanations.
What are the levels of evaluating explanations?
Evaluation can occur at the application level (real-world testing), human level (simplified tasks for laypersons), or function level (proxy tasks without humans).
What properties make explanations effective?
Expressive power, translucency, portability, and algorithmic complexity are properties that affect explanation quality.
What characteristics define good human-friendly explanations?
Good explanations are contrastive, causal, counterfactual, and tailored to the audience’s context.
What is the importance of benchmarks in XAI?
Benchmarks enable systematic comparison of explanation methods and provide standardized datasets and evaluation metrics.
What are some challenges in XAI evaluation?
Challenges include difficulty in assessing explanation quality and lack of principled guidelines for practitioners.
What are the properties of individual explanations in XAI?
The properties include accuracy, fidelity, consistency, and stability, which ensure that explanations are reliable and useful.
What does fidelity mean in the context of XAI explanations?
Fidelity refers to how closely the explanation approximates the model’s predictions, ensuring that the explanation accurately reflects the model’s behavior.
What is the significance of consistency in individual explanations?
Consistency ensures that different models provide similar explanations for the same input data, promoting reliability across models.
Why is stability important in individual explanations?
Stability means that similar instances should receive similar explanations, enhancing trust and reliability in model predictions.
What does comprehensibility mean in evaluating XAI explanations?
Comprehensibility measures how easy it is for humans to understand the explanation, taking into account the target audience’s knowledge level.
What is the purpose of evaluating explanations at the application level?
Evaluating at the application level involves real-world testing by end users to ensure that explanations are practical, useful, and understandable in actual settings.
How are explanations evaluated at the human level?
At the human level, explanations are tested with simplified tasks for laypersons to assess their understandability and usability without requiring deep technical knowledge.
What is the function level of explanation evaluation?
The function level uses proxy tasks, such as comparing explanation fidelity to model predictions on test datasets, without involving human subjects.
What is expressive power in XAI explanations?
Expressive power refers to the ability of an explanation to succinctly capture the structure of the model’s behavior, often through decision rules or visual representations.
What does translucency mean in the context of explanations?
Translucency describes the reliance of an explanation on the internal workings of the model, such as using feature weights in a linear model to clarify decision-making.
How does portability apply to XAI explanations?
Portability is the ability of an explanation technique to be applied across different model types, such as SHAP values being used with both tree-based models and neural networks.
What is algorithmic complexity in XAI evaluations?
Algorithmic complexity addresses the computational demands of generating explanations, which can impact the feasibility of using certain methods with large datasets or complex models.
Why is it important to assess the trustworthiness of explanations?
Trustworthiness ensures that explanations are reliable, unbiased, and accurately reflect the model’s decision-making, fostering confidence in the model’s predictions.
How does SHAP handle feature interaction?
SHAP captures interaction effects between features by attributing contributions to the prediction based on combinations of feature values, providing a more nuanced interpretation.
What are contrastive explanations?
Contrastive explanations highlight the differences between outcomes, explaining why one scenario occurs over another by focusing on the differing features.
What are counterfactual explanations?
Counterfactual explanations provide insights into what changes would be necessary to alter the outcome, helping to understand the conditions under which a different result would occur.
What does it mean for an explanation to be causal?
Causal explanations focus on the key causes that directly impact the prediction, highlighting the most influential features driving the model’s decision.
