Week 8 Flashcards
What ML interpretation method separates the explanations from the machine learning model?
Model-agnostic interpretation methods
What is the advantage of using model-agnostic interpretation methods over model-specific ones?
Their flexibility. The same method can be used for any type of model.
What is the disadvantage of using only interpretable models instead of using model-agnostic interpretation methods?
Predictive performance is lost compared to other ML models, and you limit yourself to one type of model.
What are two alternatives to using model-agnostic interpretation methods?
- Use only interpretable models.
- Use model-specific interpretation methods.
What is the disadvantage of using model-specific interpretation methods compared to model-agnostic ones?
It binds you to one model type and it’s difficult to switch to something else.
Name three flexibilities that are desirable aspects of a model-agnostic explanation system:
- Model flexibility
- Explanation flexibility
- Representation flexibility
Model flexibility (as an aspect of a model-agnostic explanation system)
It can work with any ML model, such as random forests and deep neural networks.
Explanation flexibility (as an aspect of a model-agnostic explanation system)
It’s not limited to a certain form of explanation. For example, linear formula and graphics with feature importances are both options.
Representation flexibility (as an aspect of a model-agnostic explanation system)
It’s able to use a different feature representation as the model being explained.
How can we further distinguish model-agnostic interpretation methods?
Into local and global methods.
What do global model-agnostic interpretation methods describe?
How features affect the prediction on average.
What do local model-agnostic interpretation methods describe?
An individual prediction.
How are global model-agnostic methods often expressed?
As expected values based on the distribution of the data.
What is the partial dependence plot?
A feature effect plot: the expected prediction when all other features are marginalized out.
When are global interpretation methods particularly useful?
When you want to understand the general mechanisms in the data or debug a model (since they describe average behavior).
PDP (abbreviation)
Partial dependence plot
PD plot (abbreviation)
partial dependence plot
What does the PDP show?
The marginal effect one or two features have on the predicted outcome of a ML model. Can show whether relationship between target and a feature is linear, monotonic or more complex.
What does xs denote in the PD function for regression?
The features for which the PD function should be plotted.
What does XC denote in the PD function for regression?
The other features (so non-xs features) used in the ML model ^f.
How does PD work?
By marginalizing the ML model output over the distribution of the features in set C, so that the function shows the relationship between the features in set S we are interested in, and the predicted outcoe.
Give the PD function for regression in the form of an expectation:
EXC[^f(xS, XC)].
Give the PD function for regression in the form of an integral:
integral sign ^f(xS, XC) dP(XC).
How is the partial function ^fS estimated?
By calculating averages in the training data, using the Monte Carlo method.
Give the partial function ^fS that is used in the PD function for regression:
^fS(xS) = 1/n sum(i=1 to n) ^f(xS,x(i)C ).
What does the partial function ^f in the PD function for regression tell us?
For given values of features S, it tells us what the average marginal effect on the prediction is.
What does x(i)C denote in the partial function ^f in the PD function for regression?
Actual features values from the dataset for the features in which we are not interested.
What is n in the partial function ^f in the PD function for regression?
The number of instances in the dataset.
What is the assumption of the PDP about the relationship between C and S?
The features in C are not correlated with the features in S.
What happens if the assumption that features in C are not correlated with features in S is violated in PDP?
The averages calculated for the PDP will include data points that are unlikely/impossible.
WHat does the PDP display for classification where the ML model outputs probabilities?
The probability for a certain class given different values for features in S.
What kind of model-agnostic method is the PDP?
A global method.
How do you calculate the partial dependence for categorical features?
Replace the feature value of all data instances with one value and average the predictions.
What does a flat PDP indicate?
The feature is not important.
How is importance of a feature defined in PDP for numerical features?
As the deviation of each unique feature value from the average curve.
What is the variable for the importance of numerical features?
I(xS)
Range rule
The way of calculating the deviation when you want a rough estimate and only know the range.
Why should the PDP-based feature importance be interpreted with care?
It captures only the main effect of the feature and ignores possible feature interactions.
Name three disadvantages of the PDP:
- It doesn’t show the feature distribution, so you might overinterpret regions with almost no data.
- Assumption of independence.
- Heterogeneous effects might be hidden (averaged out by marginalizing).
What does permutation feature importance measure?
The increase in the prediction error of the model after we permute the feature’s values, which breaks the relation between the feature and the true outcome.
When is a feature important when using permutation feature importance?
If shuffling its values increases the model error, cuz then the model relied on the feature for the prediction.
Should you compute importance on training or test data?
Since permutation feature importance relies on measurements of the model error, you should use unseen test data to prevent overfitting.
Global surrogate model
An interpretable model that is trained to approximate the predictions of a black box model.
