14. Model Risk and Governance

Question 1

Define Model Risk.

Answer 1

The potential for adverse consequences from decisions based on incorrect or misused model outputs and reports.

Question 2

What is the difference between the defective model risk and the defective application risk?

Answer 2

For a defective model risk, the model is fundamentally flawed while for the defective application risk, the model is used inappropriately.

Question 3

Name the 5 possible sources of defective model risk

Answer 3

1. Model errors
   - Model specification errors (math)
   - Model implementation errors (human error in implementation)
2. Right Model, Wrong parameters
3. User-Generated Errors (after implementation, “fat fingers”)
4. Data quality
5. Inconsistent models (between business units)

Question 4

Name the 5 possible sources of defective application risk

Answer 4

1. Right Model, Wrong application
2. Poor Communication (between model developer and users on limitations)
3. Over-Reliance on the Model
4. Failure to Heed the Model (results are downplayed or ignored)
5. Model Concentration Risk (risk of always using the same model everywhere)

Question 5

If a certain model gives an expected shortfall with a lower standard error than another model, can we say that this is a better model?

Answer 5

No, the low standard error only tells us IF the first model is appropriate, the standard error of the ES calculation would be lower.

Question 6

What is model governance?

Answer 6

Model governance refers to the process and policies established to ensure that models used by a firm are developed, reviewed, and maintained consistently with the risks involved and with the firm’s risk appetite.

Question 7

What are the 5 major functions of model risk governance?

Answer 7

1. Model lifecycle management
2. Model inventory
3. Model risk materiality
4. Internal audit
5. Model documentation

Question 8

How can you mitigate the model risk of misspecification and/or coding errors?

Answer 8

Through strong model governance, acquisition of highly qualified staff and with rigorous vetting and approval requirements

Question 9

How can you mitigate the risk of misuse of model results?

Answer 9

1. Strong training of users
2. Model users are representad at the design stage
3. Good design of model output and easy to interpret
4. Creating feedback channels
5. Ensuring models cannot be deployed outside their original purpose

Question 10

How can you mitigate the risk of model and parameter uncertainty?

Answer 10

• Avoid behavioural risk by ensuring that any stakeholder with an interest is not involved in vetting or approval.
• Stress test results based on realistic variation in parameters.
• Use Bayesian framework to incorporate model and parameter uncertainty into calculations.

Question 11

What is knightian uncertainty?

Answer 11

Knightian uncertainty is a lack of any quantifiable knowledge about some possible occurrence, as opposed to the presence of quantifiable risk.