Chapter 8 - Model Risk Flashcards
what has caused model risk to increasingly be used?
Machine Learning (ML) and algorithms are becoming more complex, therefore effective risk model management has become top priority.
Why is model risk an operational risk?
- the model may have errors and produce inaccurate errors
- the model may be used incorrectly or inappropriately
benefits of modelling?
- enables an unclear and complicated reality to be represented in a way which enables sound investment decisions
- speeds-up decision making
- enables ‘what-if’ analysis to be performed.
limitations of modelling
- cant ‘re-use’ outputs
- shape of any underlying distribution used by the model
- relationship between past and future
- state of business environment at the point when the model was designed.
- focuses on maximum confidence level
benefits and limitations of modelling with AI
benefits - optimise business operations, automated system, efficient , productive, profitable
Limitations - biased, predictable results, require large amounts of data to be trained, complex, expensive
Know the major models utilised: Operational Risk scenario modelling
steps - workshop and stressed
This approach helps calculate the required level of capital.
Steps
- Workshops are conducted with senior staff to estimate how often risk happen, size of loss, how severe loss could be.
These estimates are then stressed, meaning they are adjusted to reflect rare, but serious events: this helps, look at possibility of actual loss being larger than expected, risk of multiple events occurring.
How does operational risk loss events distributed?
- they are not normally distributed, and instead follow a distribution that exhibits fat tail characteristics
To model this accurately, lognormal distributions are used (which account for those rare, large losses used)
what do firm rely on to estimate how much capital is needed to handle severe losses?
Monte Carlo simulations
- process which uses random scenarios to simulate possible losses and their likelihood.
- the process randomly samples from lognormal curve, which helps determine how often extreme losses could happen.
what is Poisson distribution? what does it estimate, what does it reflect
estimates how often a specific risk event might occur.
this distribution reflects the randomness of rare events.
Deriving the impacts: Lognormal curve and random impacts, sorting and confidence levels, individual scenario sorting, diversification effect.
- using estimates from workshops (typical and extreme), a lognormal curve is created to represent the possible losses for each scenario. a random point is chosen from this curve.
sorting and confidence levels
- total losses are calculated ranked from smallest to largest
- for example, to find 99.5% confidence level, they look at 99,500th highest.
Individual scenario sorting
-impacts for each scenario are ranked separately, this means sorting each scenario impact and selecting 99.5% value for that specific scenario
Diversification effect:
- not all scenarios happen same time, if model assumes scenarios occur independently, overall risk may be lower. This is called portfolio effect, which reflects spreading risk across different events can reduce total exposure.
Therefore firms can estimate both individual scenarios and overall portfolio risk.
Credit risk modelling: Credit VaR - example, what it uses, what is also published, what it enables us to do/
Example - attempt to predict the value of a bond in one year time, according to probability of moving from one credit rating agency to another.
Credit Migration probabilities published, due to it being probability (confidence level) , standard deviation also published, enables us to plot distribution and read off value.
Market risk modelling: Model VaR
Expresses the maximum market loss that can occur with a specified confidence over specified period.
Due to uncertainty = includes the level of confidence levels
One method to work out is historical simulation
steps
- identify risk factor
- select sample of historic risk factor
- systematically apply each of daily changes to current value of risk factor
- list out all resulting portfolio values
another method
Liquidity-at-risk (LaR)
- funding amounts brackets
- net funding mounts are counted and put in brackets
- brackets arranged as distrubution
- provides an idea of likely funding requirements over time at different confidence levels.
what does the federal reserve document Supervisory guidance on model risk management cover?
- roles and responsibility of different parts of organisation including board, management
- use of policies and procedures
the roles of board of directors and seniors
Establish a firm-wide framework for model risk management
Should include: model development, implementation, use and validation.
Should ensure: Level of model risk is within risk appetite. This sets tone for whole org.
Duties of senior management?
- Establish adequate policies and procedures
- assign competent staff with model ownership
- oversee model development and implementation
- establish model risk controls
- recognise need to creation of provisions for trade
internal audit role
if certain audit staff perform day-to-day validation, what happens?
access the overall effectiveness of model risk management framework.
If certain internal audit staff perform day-to-day validation, they should not be involved in assessment of overall model framework
external resources
if they are used…
Although model risk management is internal process, firm may decide to get external resources to help.
If they are used, the firm must specify the activities to be conducted in clear written way and agreed-upon scope of work.
If they are used, internal party from firm should evaluate the results of design by external.
If they are used, the firm should have a contingency plan
Polices, procedures, and documentation
must cover, must emphasise, must include
the board must do
policy must cover:
- model risk definition, assessment of model risk, acceptable practises, appropriate model validation, governance and controls over risk management process.
Policy MUST emphasise testing and analysis, and promote the development of targets.
Also must include, prioritisation, scope, and frequency of validation activities. Also the detail for validation of vendor models and third-party products. Also the maintenance of documents
The board must periodically review!
