Week 8

Question 1

What ML interpretation method separates the explanations from the machine learning model?

Answer 1

Model-agnostic interpretation methods

Question 2

What is the advantage of using model-agnostic interpretation methods over model-specific ones?

Answer 2

Their flexibility. The same method can be used for any type of model.

Question 3

What is the disadvantage of using only interpretable models instead of using model-agnostic interpretation methods?

Answer 3

Predictive performance is lost compared to other ML models, and you limit yourself to one type of model.

Question 4

What are two alternatives to using model-agnostic interpretation methods?

Answer 4

1. Use only interpretable models.
2. Use model-specific interpretation methods.

Question 5

What is the disadvantage of using model-specific interpretation methods compared to model-agnostic ones?

Answer 5

It binds you to one model type and it’s difficult to switch to something else.

Question 6

Name three flexibilities that are desirable aspects of a model-agnostic explanation system:

Answer 6

1. Model flexibility
2. Explanation flexibility
3. Representation flexibility

Question 7

Model flexibility (as an aspect of a model-agnostic explanation system)

Answer 7

It can work with any ML model, such as random forests and deep neural networks.

Question 8

Explanation flexibility (as an aspect of a model-agnostic explanation system)

Answer 8

It’s not limited to a certain form of explanation. For example, linear formula and graphics with feature importances are both options.

Question 9

Representation flexibility (as an aspect of a model-agnostic explanation system)

Answer 9

It’s able to use a different feature representation as the model being explained.

Question 10

How can we further distinguish model-agnostic interpretation methods?

Answer 10

Into local and global methods.

Question 11

What do global model-agnostic interpretation methods describe?

Answer 11

How features affect the prediction on average.

Question 12

What do local model-agnostic interpretation methods describe?

Answer 12

An individual prediction.

Question 13

How are global model-agnostic methods often expressed?

Answer 13

As expected values based on the distribution of the data.

Question 14

What is the partial dependence plot?

Answer 14

A feature effect plot: the expected prediction when all other features are marginalized out.

Question 15

When are global interpretation methods particularly useful?

Answer 15

When you want to understand the general mechanisms in the data or debug a model (since they describe average behavior).

Question 16

PDP (abbreviation)

Answer 16

Partial dependence plot

Question 17

PD plot (abbreviation)

Answer 17

partial dependence plot

Question 18

What does the PDP show?

Answer 18

The marginal effect one or two features have on the predicted outcome of a ML model. Can show whether relationship between target and a feature is linear, monotonic or more complex.

Question 19

What does x_s denote in the PD function for regression?

Answer 19

The features for which the PD function should be plotted.

Question 20

What does X_C denote in the PD function for regression?

Answer 20

The other features (so non-x_s features) used in the ML model ^f.

Question 21

How does PD work?

Answer 21

By marginalizing the ML model output over the distribution of the features in set C, so that the function shows the relationship between the features in set S we are interested in, and the predicted outcoe.

Question 22

Give the PD function for regression in the form of an expectation:

Answer 22

E_XC[^f(x_S, X_C)].

Question 23

Give the PD function for regression in the form of an integral:

Answer 23

integral sign ^f(x_S, X_C) dP(X_C).

Question 24

How is the partial function ^f_S estimated?

Answer 24

By calculating averages in the training data, using the Monte Carlo method.

Question 25

Give the partial function ^f_S that is used in the PD function for regression:

Answer 25

^f_S(x_S) = 1/n sum(i=1 to n) ^f(x_S,x⁽ⁱ⁾_C ).

Question 26

What does the partial function ^f in the PD function for regression tell us?

Answer 26

For given values of features S, it tells us what the average marginal effect on the prediction is.

Question 27

What does x⁽ⁱ⁾_C denote in the partial function ^f in the PD function for regression?

Answer 27

Actual features values from the dataset for the features in which we are not interested.

Question 28

What is n in the partial function ^f in the PD function for regression?

Answer 28

The number of instances in the dataset.

Question 29

What is the assumption of the PDP about the relationship between C and S?

Answer 29

The features in C are not correlated with the features in S.

Question 30

What happens if the assumption that features in C are not correlated with features in S is violated in PDP?

Answer 30

The averages calculated for the PDP will include data points that are unlikely/impossible.

Question 31

WHat does the PDP display for classification where the ML model outputs probabilities?

Answer 31

The probability for a certain class given different values for features in S.

Question 32

What kind of model-agnostic method is the PDP?

Answer 32

A global method.

Question 33

How do you calculate the partial dependence for categorical features?

Answer 33

Replace the feature value of all data instances with one value and average the predictions.

Question 34

What does a flat PDP indicate?

Answer 34

The feature is not important.

Question 35

How is importance of a feature defined in PDP for numerical features?

Answer 35

As the deviation of each unique feature value from the average curve.

Question 36

What is the variable for the importance of numerical features?

Answer 36

I(x_S)

Question 37

Range rule

Answer 37

The way of calculating the deviation when you want a rough estimate and only know the range.

Question 38

Why should the PDP-based feature importance be interpreted with care?

Answer 38

It captures only the main effect of the feature and ignores possible feature interactions.

Question 39

Name three disadvantages of the PDP:

Answer 39

1. It doesn’t show the feature distribution, so you might overinterpret regions with almost no data.
2. Assumption of independence.
3. Heterogeneous effects might be hidden (averaged out by marginalizing).

Question 40

What does permutation feature importance measure?

Answer 40

The increase in the prediction error of the model after we permute the feature’s values, which breaks the relation between the feature and the true outcome.

Question 41

When is a feature important when using permutation feature importance?

Answer 41

If shuffling its values increases the model error, cuz then the model relied on the feature for the prediction.

Question 42

Should you compute importance on training or test data?

Answer 42

Since permutation feature importance relies on measurements of the model error, you should use unseen test data to prevent overfitting.

Question 43

Global surrogate model

Answer 43

An interpretable model that is trained to approximate the predictions of a black box model.