CIA Flashcards
purposes of stress-testing
- risk: identify & control risk
- complement: provide a complement to other risk management tools and simulate shocks
- cap: support capital management
- liq: improve liquidity management
describe the stress-testing purpose: ‘risk identification & control’
- risk identification: identify concentrations & interactions of risks
- risk control: adjust individual portfolios or overall business strategy
describe the stress-testing purpose: ‘complementing other tools’
- test statistical models used to determine VaR
- simulate shocks to test model robustness to economic changes
describe the stress-testing purpose: ‘supporting capital management’
identify severe events and/or compounding events that impact capital requirements
describe the stress-testing purpose: ‘improving liquidity management’
assess liquidity profile and adequacy of buffers for institutional & market-wide stresses
key elements of FCT
(BACRO):
- base scenario: must develop a base scenario and usually the insurer’s current business plan
- adverse scenario: must develop multiple adverse scenarios (i.e, covid, climate change, etc)
- corrective action: identification and analysis of corrective management actions to mitigate risks
- report: submit recommendations to management and the board of directors or chief agent
- opinion: appointed actuary signs an opinion regarding the financial condition of the insurer
key metrics that must be understood when performing FCT
- regulatory capital minimum
- insurer’s internal target capital requirements determined by ORSA
identify the ‘preliminary’ step and the ‘extra’ step in addition to ‘BACRO’ when performing FCT
- preliminary: review financial position at year-end for each year in historical period
- extra: identify possible regulatory action
what is a review of operations and financial position
- review balance sheet, statement of income, and source of earnings for an appropriate number of years
what is the forecase period for FCT
the forecast period should be long enough to capture
- risk emergence
- financial impacts
- ripple effects
- corrective action
-> generally 3-5 years although there is no minimum and should also be consistent with ORSA
how do you determine the materiality standard for FCT
FCT sets the materiality standard with management input and by specifically considering:
- size of insurer
- financial position
- nature of regulatory test
define base scenario
a set of assumptions on risk factors that are consistent with the business plan over the forecast period
define adverse scenario
a scenario that is developed by stress-testing assumptions used in the business plan, look specifically for risk factors that threaten financial condition
define solvency scenario
a plausible adverse scenario that has a non-trivial probability of occuring
- should fall above the 95th percentile on the loss distribution
- or possibly as high as the 99th percentile and beyond depending on circumstances
define going-concern scenario
an adverse scenario that is more likely and/or less severe than a solvency scenario
- should fall above the 90th percentile on the loss distribution
- could include risks not considered in solvency scenarios
define ripple effect
- an event that occurs when an adverse scenario triggers a change in 1 or more inter-dependent assumptions
- can include policyholder actions, management’s routine actions, regulatory actions
i.e, a ripple effect of an earthquake may be loss of reinsurance
what is a corrective management action
an action management takes to mitigate adverse ripple effects
what is an integrated scenario for FCT
a scenario created by combining two or more risk factors to produce a new plausible adverse scenario
i.e, combine a low-probability scenario with a higher-probability adverse scenario
considerations in the development of a climate change integrated adverse scenario
(PTL):
- physical risk -> frequency and severity of wildfires, floods, wind events, rising sea levels
- transition risk -> due to economic shift to greener technologies
- liability risk -> exposure to climate-related litigation
examples of IFRS 17 measurement features to consider for FCT scenarios
IFRS 17 liabilities:
- make no provision for default risk
- do not reflect the benefit of discounting arising from deferred tax assets
- include the CSM when evaluating solvency scenarios
check CIA.FCT-1 wiki for more possible answers
key elements that an FCT model should reproduce
- balance sheet: assets, liabilities, retained earnings,…
- income statement: revenue & expenses
- regulatory measures of capital adequacy: MCT ratio, and possibly others like BCAR or MSA ratios
- sources of earnings: detail on sources of premium and investments
what is the recommended loss distribution for a going-concern scenario
90th percentile -> 95th percentile
what is the recommended loss distribution for a solvency scenario
95th percentile -> 99th percentile
how does an actuary validate an FCT model on an accounting basis
verify: statement of income = cash flows + change in balance sheet items
how does an actuary validate an FCT model in a static environment
base scenario should show continuity of results from year-to-year cash, liabilities, surplus,…
how does an actuary validate a new FCT model or model update
- new model: run with data at (t-1) & compare to actual data at (t) should be close
- model update: do a retrospective test compare prior base scenario projection to current data
how does an actuary validate an FCT model in a changing environment
- ask: does model properly quantify changes in results under different assumptions
- compare: 2 adverse scenarios magnitude & direction on change should be consistent with assumptions
when is a stochastic FCT model appropriate
- when risk distributions are easily inferred
- capital market risks
stochastic models generate multiple randmoized scenarios and models random movements in key risk drivers like:
- interst rates
- equity market returns
- claim frequencies and severities
- inflation
- policyholder behaviors
stochastic models are used for solvency 2, LICAT and IFRS 17, help insurers determine captial adequacy, reinsurance needs and strategic risk management
when is a deterministic FCT model appropriate
- when risk distributions are not easily inferred
- actuary then select scenarios based on historical experience, credibility of data
deterministic models analyzes only a few well-defined scenarios and project future financial conditions based on fixed, specific assumptions like interest rates, claim experience and market performance
deterministic models are used for stress-testing and solvency analysis as required by regulators (i.e, OSFI)
what is a combination stochastic/deterministic model
when results of a stochastic model are used to derive a deterministic scenario that reproduces the stochastic results
OR
A combination stochastic/deterministic model is used when stochastic modeling (which incorporates randomness and multiple scenarios) is run first, and the results are then used to create a deterministic scenario that closely replicates the key outputs of the stochastic model.
how are ripple effects in a FCT analysis modeled
- automatically: by computer model
- manually: by actuary based on knowledge of situation
what are some considerations in FCT model segmentation
- management: segment around management structure
- product: smallest subdivision - may combine similar products
- investment: asset categories
identify an important IFRS 17 concept to consider when creating an FCT scenario
CSM for GMA and VFA approaches
“GMA = general measurement approach
VFA = variable fee approach (specifically desgined for direct participating contracts and it directly links insurers’ profits to the performance of the underlying assets)”
does PAA approach have a CSM component? Explain
no, but PAA requires a loss component for onerous contracts and LC cannot be offset by future profits so ‘level of aggregation’ is an important consideration
identify aspects of IFRS 17 that should be considered when creating FCT scenarios
- the impact of adverse scenarios on onerous groups is not absorbed by the CSM - it will be reflected in earnings immediately
- modelling will need to capture the behavior of groups of contracts rather than individual contracts
- groups of reinsurance contracts are modeled separately from the underlying primary insurance contracts issued
- the business volume forecast requires sufficient granularity to model the timing of recognition of new cohorts
purpose of an FCT report
communication to BoD:
- identify risks to an insurer’s financial condition
- identify ways to mitigate and reduce risk
audiences for FCT report
- BoD: prefers an interpretive summary vs. a detailed statistical report
- management: receives a more detailed report
- regulator: focuses on solvency issues
possible types of opinion that AA can include in the FCT report
- satisfactory
- satisfactory subject to appropriate corrective action
- not satisfactory
when can AA report that financial condition of an insurer is satisfactory
- under the base scenario insurer meets its internal target capital ratio as determined by ORSA
- under the going-concern scenarios -> insuer meets the regulatory minimum capital ratios
- under solvency scenarios -> must have assets > liabilities
how many adverse scenarios should an FCT report include
- at least 3, 1 going-concern scenario and 2 solvency scenarios
- they should also be chosen from multiple risk cateogries
P&C risk categories
(F-PIP-REAGOR-C^2):
- frequency&severity
- policy liabilities
- inflation
- premiums
- reinsurance
- expense risk
- asset risk
- government risk
- off-balance-sheet risk
- related company risk
- climate risk
- cyber risk
common ripple effects
think about the ripple effects from COVID
- higher LR
- loss of reinsurance
- post-event inflation
- forced sale or liquidation
- mix shift
- policyholder actions
- regulatory action
loss ratio goes up because of higher severity and elevated inflations, also it caused some business shut down which leads to loss of reinsurance for commercial, there are shifts in mix of business of the drivers, and policyholders drive more wild due to fewer cars on the road, and regulator intervene for covid relief measures
common corrective management actions
- tighten U/W guidelines
- raise rates
- review reinsurance
- sell assets
- review mix of business
2 more corrective management actions only applicable in certain situations
- suspend dividend payments
- reduce capital transfers to parent or home office
a ripple effect and management action for the ‘cyber risk’ adverse scenario
- data breach
- invest in cybersecurity and IT infrastructure
possible ripple effects and corrective management actions for: catastrophe
ripple effects:
- post-event inflation
- loss of reinsurance
corrective management:
- raise rates
- review reinsurance
possible ripple effects and corrective management actions for: a change in asset risk
ripple effects:
- forced sale or liquidation
- signifcant +/- change in cash flows
corrective management:
- sell assets
- change investment strategy
a change in asset risk can be:
- significant changes in yield curve
- decrease in return on equity investments
methods for selecting adverse scenarios
- percentiles if a loss distribution is available
- reverse stress-testing
describe the method of reverse stress-testing in an FCT analysis
- start by considering a specific adverse scenario where the insurer’s surplus becomes negative (surplus = assets - liabilities)
- work backwards to find the risk factors required to produce that scenario
- determine if it’s plausible for risk factors ofthe insurer’s current financial position to deteriorate to that degree
-> if yes, then this adverse scenario may be a solvency scenario
should the actuary integrate the FCT report with the ORSA report or keep them separate
use judgement, may produce 2 independent reports or 1 integrated report
considerations in supporting integration of FCT and ORSA
- FCT uses ICT ratios developed by ORSA and these target ratios may develop over the timeframe of a projection
- ORSA is useful in assessing adverse scenarios
- may be more efficient to integrate the reports, both require data collection and similar types of analysis, both may be released at the same time
- a single integrated report may be better for the end user
challenges regarding integration of FCT and ORSA
- oversight: AA is responsible for FCT whereas the board and senior management is responsible for ORSA
- different methodology: FCT follows a prescribed regulatory basis while ORSA reflects own models and assumptions
- staff responsible: different for FCT vs. ORSA and coordination may be costly
type of duration
Macaulay duration, modified duration, effective duration
what is the formula relating Macaulay and modified duration
modified duration = Macaulay duration / (1 + discount rate)
what is the difference between modified duration and effective duration
- effective duration accounts for situations where a change in interest rates changes the cash flows
- modified duration doesn’t account for this
describe the concept behind ‘modified duration’
modified duration is the approximate % change in PV(cash flows) from a 100bps change in interest rate assuming no change in cash flows
describe the concept behind ‘effective duration’
effective duration recognizes that a change in interest rate may also cause a change in cash flows
things to consider in calculating duration for liabilities
- consistency of assumptions: assumptions for duration calculation should be consistent with the discounting calculation from the valuation
- duration calculation by LOB: use same payout patterns as for discounting then total duration is a weighted average with weights by APV by LOB
- duration calculation on a combined LOB basis: use effective duration
- when interest rate is small: modified duration and effective duration are approximately the same
is it permissible for the actuary to rely on an investment specialist for the calculation of asset duration? Explain
- yes, the actuary would be the enquiring professional, and the investment specialist would be the responding professional
- the actuary must review the investment specialist’s work for methodology and reasonableness
the key principles of risk transfer assessment
- use quantitative and/or qualitative approaches depending on information available
- use professional judgment
- consider overall agreement all verbal & non-verbal agreements
- check risk transfer at inception of contract and re-check whenever changes affect future cash flows
in a risk transfer contract, what is included in the ‘overall agreement’
contract, amendments, verbal agreements, other written docs
when should existence of risk transfer be rechecked
- at inception
- when contract change significantly alters expected future cash flows
changes to reinsurance contract that would trigger recheck of risk transfer
revision to premiums or coverage levels other than linear increase/decrease of quota share
changes to reinsurance contract that would not trigger recheck of risk transfer
events that are part of the normal course of the contract
i.e, build-up of a claim fluctuation reserve
what should actuary do prior to recheck risk transfer
check whether previous reinsurance assessment is still applicable
qualitative assessment of risk transfer
- is it obvious that the cedant’s financial interests are protected
- don’t focus on probabilites: coverage for a low frequency/high severity passes if contract is arms-lenth and/or no risk-limiting features (specifically for earthquake insurance)
does ‘reasonably self-evident’ apply to this cat treaty (earthquake insurance)
- difficult to assess since we know nothing about the treaty
- it’s a low-frequency/high severity situation so it could pass the ‘reasonably self-evident’ test if the premium and coverage are appropriate
2 broad categories of risk-limiting contract features
- terms set in advance
- experience-based renewals (EBR)
types of terms-set-in-advance risk limiting features
- adjustability of reinsurance premiums of commissions
- pre-set limits on timing of loss payments from reinsurer to insurer - removes timing risk
- counterparties ceding back to original cedant
examples of experience-based renewals risk limiting features
- future terms based on past experience & reinsurer guaranteed to recover losses
- forced renewals if the contract is in deficit
define side agreement
agreement between cedant, reinsurer not directly incorporated into contract - may obscure intent of contract
define mirroring + comment
- definition: cedant & reinsurer carry similar liability estimate for the ceded claims
- comment: it is appropriate for cedant & reinsurer actuaries to confer on large losses
considerations in estimating a credit provision for a counterparty
- best rating of reinsurer
- expertise of reinsurer in relevant lOBs
- diversification of reinsurer
- disputes: history of claims disputes
define materiality
an omission/under-statement/over-statement is material if the actuary expects it to materially affect the user’s decision-making or reasonable expectations
what materiality is not
- materiality is not a range of reasonable values around actuarial estimate
- materiality is not inherent uncertainty in an actuarial estimate
identify the main consideration in setting a materiality level
specify:
- use of work, intended users
- there is no obligation to communicate with other than intended audience
identify circumstances where the materiality level should change
- when an external benchmark is approached (i.e, regulatory action level)
- otherwise, it should be consistent over time and between valuations
identify charactersitics of an insurance company that may affect materiality
(F-STARS):
- financial strength
- size of entity
- type of business
- access to capital
- net retention
- stage of organization’s life cycle
identify a metric to test materiality for regulatory or solvency purposes
- statutory surplus
- solvency benchmark ratio
identify a metric to test materiality for appraisals
- net worth
- net income
- EPS(earnings per share)
identify a metric to test materiality for general purpose financial statements
- statutory surplus
- net income
identify which application has a less rigorous materiality level, DCAT or Valuation
DCAT is less rigorous so materiality standard is higher
DCAT:
- used for surplus in scenario-testing
Valuation:
- this impacts net income, which is more important
- need to detect smaller deviations here
considerations for extent of disclosure materiality
- sophistication of user
- important of concept to user
- complexity of concept
possible actions of report-writer based on materiality
- include them?
- refine them? (if it’s sufficiently accurate)
- disclose them?
briefly describe when an actuarial approximation is appropriate
an approximation is appropirate if it does any of these without affecting the result:
- reduces cost
- reduces time
- improves control
approximation should strike a balance between benefit and effort
an example of an approximation that is appropriate
for a multi-billion dollar line of business: spend 99 cents to reduce error in your reserve estimate by 3.14%
an example of an approximation that is not appropriate
for a significant line of business run-off: give up your first-born to reduce the error in your reserve estimate by 2.718%
examples of quality assurance processes
- check valuations
- validate models
- redo the work
- peer review
considerations in determining what quality assurance processes to perform
- purpose, complexity, significance of work
- vulnerability to error
- expectations of user
- legislative requirements for peer review
identify items an actuary should consider before using another person’s work
- qualifications of the other person
- regualr communication & info-sharing with the other person
- awareness by the other person of how the work is being used
identify the purpose of CIA/CICA joint policy statement
JPS discusses: (RID)
- reliance of auditor & actuary on each other’s work
- interaction between auditor & actuary
- disclosure of responsibilities of auditor & actuary
define ‘enquiring professional’
a professional who relies on the work of another in the course of their own work
define ‘responding professional’
a professional whose work is being used by another
identify management’s responsibility under JPS regarding financial statements
overall responsibility for F/S may include amounts determined by the actuary
actuary’s responsibility under JPS regarding financial statements
- actuary is responsible for ‘RelSuff’ of data
- actuary is not responsible for data integrity or controls
auditor’s responsibility under JPS regarding financial statements
- auditor is responsible for integrity of financial statements (financial position, operations, cash flows)
- auditor is not responsible for the actuarial valuation
communication requirements for enquiring professional with responding professional
- notify responder that work is being used
- notify responder regarding needs (materiality, subsequent events, timing)
- request confirmation of responder’s appointment, professional standing, observance to standards
communication requirements for responding professional with enquiring professional
- confirm that work will be done
- confirm appointment, professional standing, observance of standards
- notify of any issues in meeting enquiring professional’s needs
4 types of actuarial reports according to CSOP 1700
- external report:
- formal & detailed
- range of appropriate reports is narrow - internal report:
- may be formal & detailed or informal & abbreviated depending on use & user
- range of appropriate reports is wide - oral report:
- useful to internal user
- disadvantage vs. internal report is no written record - summary report:
- a simplified way to communicate actuary’s analysis
- may be part of an external of internal report
items included in an external report according to CSOP 1700
- name of client
- description of work, use, user
- statement that it may not be appropriate for other uses
- statement about whether accepted actuarial practice was used
- assumptions and justifications
- methods
- reservations or qualifications
scope of CSOP 2600 regarding derivation & recommendation of dates
within scope: deviation of rates
not within scope: recommendation of rates
how should an actuary incorporate inflation assumptions in a reserve analysis
- consult with underwriters, business analysts, fraud detection experts, claim adjusters to understand whether inflation has transpired in claim payments, and is accounted for in case reserves
- consult with CPI
- perform sensitivity analysis with varying inflation assumptions to assess the sensitivity of reserve estimates
why may the development method not work for long-tailed with sudden changes in inflation
the effect of inflation on recent development periods may emerge more quickly for short-tailed lines but more slowly for long-tailed lines
what is the ongoing role of actuaries concerning COVID19
- follow guidance from regulatory bodies like OSFI
- monitor legal actions related to COVID19 that might impact valuations
what specific requirement does OSFI have for insurance companies related to COVID19
report statistics and impacts in Appointed Actuary’s Report
how are actuaries approaching the evaluation of trends and metrics affected by COVID19
take a longer-term view to cover pre- and post-pandemic impacts
what specific factors are easier to isolate concerning COVID19’s impact on policy liabilities
- premium reductions
- refunds
- cost of material and labour
how will IFRS17 affect the valuation of actuarial liabilities
discounting:
- use a yield curve instead of a single discount rate
- no more PfAD for interest/discount rate ==> risk adjustment for financial risk is implicitly included in yield curve
risk adjustment:
- adjust PV(future cash flows) for uncertainty in amount&timing related to non-financial risk
aggregation:
- create portfolios of similar risks that are managed together: onerous risks, non-onerous risks, remaining risks
measurement:
- may use PAA for measuring remaining liabilities
- PAA is a simplified version of BBA(building block approach) and can only be used under certain conditions
reporting:
- carrying amounts for 4 groups must be shown separately in financial statements:
-> insurance contracts that are assets/liabilities
-> reinsurance contracts that are assets/liabilities
define yield curve
a yield curve is a function relating yield of fixed-interest securities to the length of time to maturity
handling the time value of money in claim runoff
- discounting: discount the paid&unpaid amounts at time t back to t-1
- subtraction: subtract investment income earned during CY t on assets that support the liabilities easier than discounting
define calculation date
effective date of calculation
define report date
date on which the actuary completes the report on his/her work
define report
actuary’s oral or written communication to users about his/her work
define subsequent event
an event of which an actuary first becomes aware after the calculation date but before the corresponding report date
define adjusting event + example
- event after calculation date that provides evidence of condistions existing at calculation date: results in adjustments
- i.e, reinsurer insolvency after calculation date that was due to gradual deterioration occuring before calculation date
define non-adjusting event + example
- event after calculation date indicative of conditions arising after calculation date: no adjustments
- i.e, reinsurer insolvency due to catastrophe
subsequent event case: catastrophic (1998 ice storm)
Facts(timeline):
- event: 1/5/1998
- actuary became aware 1/5/1998, the same day as the event
- subsequent event? Yes, actuary became aware after the calculation date 12/31/1997 and before the report date(several weeks after the event)
Branch: middle
Action:
- error? no, next
- when? after the calculation date, next
- different? yes, after the calculation date, next
- purpose? report on entity as it was
-> inform only
comments:
- the ice storm doesn’t make the entity different retroactively
- the purpose of the actuary’s work was to report on the entity as it was
- but note that the premium liabilities could have been understated
what is the prime consideration to apply the subsequent event tree
is the event material?
- if not material then usually no action is required
- but sometimes it’s still beneficial to disclose the event
subsequent event case: late reported claims
Facts:
- event: 11/20 (case reserve increasing by the ceding insurer)
- actuary became aware 1/12
- subsequent event? Yes, actuary became aware after the calculation date (12/31) and before the report date (several weeks after 1/12/1998)
Branch: middle
Action:
- error? no, next
- when? before the calculation date
-> reflect
comments:
- this situation often arises for reinsurers
- it is not the same as a data error or missing claims
subsequent event case: Alberta Minor Injury Cap
Facts:
- event: 2/8/2008, $4,000 Alberta Minor Injury Cap struck down
- actuary became aware 2/8/2008, the same day
- subsequent event? Depends on the insurer’s report date
Branch:
- if the report date is after 2/8/2008, then it’s subsequent event, go middle
- if the report date is before 2/8/2008, then go right
Action(if subsequent):
- error? no, next
- when? after the calculation date, next
- different? yes
-> reflect
Action(if not subsequent):
- if material, withdrawl/amend
- if not material, inform only
comments:
- this case is complicated because some insurers would have completed their reports before 2/8/2008, but some would not
subsequent event case: reinsurer failure
Facts:
- event: 1/15
- actuary became aware 1/15, the same day
- subsequent event? Yes, actuary became aware after calculation date 12/31 and before report date (several weeks after 1/15)
Branch: middle
Action:
- error? no, next
- when? after calculation date, next
- different? yes, before calculation date (failure provided evidence of prior deteriorating conditions)
-> reflect
comments:
- action depends on cause of failure: here reinsurer failure had been building prior to calculation date, failure after calculation date simply provided further evidence of condistions existing prior to calculation date
- if failure was due to a catastrophe, path through decision tree would be EWDP, and final action will be inform only
subsequent event case: stock market drop
Facts:
- event: 1st week of Jan, big drop in stock market
- actuary became aware 1st week of Jan
- subsequent event? Yes, actuary became aware after the calculation date 12/31 and before report date (several weeks after 1st week of Jan)
Branch: middle
Action:
- error? no, next
- when? after calculation date, next
- different? yes after the calculation date, next
- purpose? report on entity as it was
-> inform only
comments:
- main issue is whether the market drop provided evidence of conditions in existence prior to calculation date or not
- it was decided that it didn’t different from the ‘reinsurer failure’ example
subsequent event case: missing claims
Facts:
- event: before 6/30 claims database was missing claims
- actuary became aware 8/5
- subsequent event? Yes, actuary became aware after calculation date 6/30 and before the report date (after 8/5)
Branch: middle
Action:
- error? Yes
-> reflect
comments:
- if actuary had become aware after the report date, this wouldn’t be a subsequent event, we would then be on the right branch of the decision tree and ask these questions instead:
-> would the event be reflected if it had been a subsequent event? yes, next
-> does the event invalidate the report? yes, withdraw or amend report
subsequent event case: change in industry benchmark
Facts:
- event: 7/15, new industry menchmark released
- actuary became aware 7/15
- subsequent event? Yes, actuary became aware after quarterly calculation date 6/30 and before report date (several weeks after 7/15)
Branch: middle
Action:
- error? No, next
- when? after the calculation date, next
- different? event doesn’t make any entity different - LDFs don’t change much from quarter to quater
-> final action: nothing
Comments:
- this situation could arise from a new company without its own credible data and must rely on industry benchmarks
subsequent event case: pandemic
Facts:
- event: 3/31 pandemice was declared
- actuary became aware 3/11
- subsequent event? Yes, actuary became aware after the calculation date 12/31 and before the report date
Branch: middle
Action:
- error? No, next
- when? After the calculation date, next
- different? yes, after the calculation date, next
- purpose? report entity as it was
-> inform only
Comments:
- provide potential impacts of the impact of the pandemic either qualitatively or quantitatively, when situation for classification is unclear, discuss with auditor
identify circumstances under which a subsequent event must be accounted for
- if the event is material
- if the event reflects an error
- if the event makes the entity different relative to the timing/purpose of the actuary’s report
define model
a practical representation of relationships among entities using FEMS(financial, economical, mathematical, statistical) concepts
elements of a model
(SIR):
- model specification
- model implementation
- model run
define model specification
a description of the parts of a model and their interactions includes data, assumptions, methods, entities, events
define model implementation
the systems that perform the calculations(computer programs, spreadsheets,…)
define model run
the inputs/outputs of the implementation
define model risk
the risk that the user will draw inappropriate conclusions due to shortcomings of the model or its use
main difference between calculation and model
a model requires more documentation(how it was chosen, how it’s used)
why there’s always risk in using a model
because a model is a simplification of reality
how can model risk be measured
- severity of model failure
- likelihood of model failure
considerations in assessing the severity of model failure
- financial significance (higher severity if estimating a major balance sheet item)
- importance of model (severity is lower if multiple models are being used)
- frequency of use of model (severity is higher if model is used frequently)
considerations in assessing the likelihood of model failure
- complexity (higher complexity means higher likelihood of misuse of the model)
- expertise(non-expert users may not understand model limitations)
- docs (bad docs means high likelihood of model failure)
- testing (inadequate testing means high likelihood of model failure)
does the actuary have more control over the severity or likelihood of model failure
more control over likelihood:
- choose a more reliable model within the actuary’s control
- test the model more thoroughly within the actuary’s control
steps an actuary should take before using a new model
common sense + what to do about elements of model
review specification, validate implementation, deal with limitations, keep documentation
describe what an actuary does when reviewing model’s specifications
verify DAMs:
- your data fits the models requirements
- methods are sound
- assumptions are appropriate
describe what an actuary does when validating a model’s implementation
- compare with other tested models
- maintain a set of test cases
- backtesting (testing with historical data where you already know the answer)
- run an entire live file through successive versions of the model for models with a higher risk-rating
- peer review of testing procedure
describe what an actuary does when dealing with a model’s limitations
understand the range of uses for which the model was designed and tested
describe what an actuary should include when documenting a model
- how the model was chosen
- how it was tested
- what are its limitations
what is an important tool for validating models
a model’s risk rating
how should an actuary evalulate an existing model that’s being used in a new way
- check the initial model as properly validated
- review limitations in the new application that may not have been relevant in the intial application
how should an actuary evaluate a model approved for use by others
actuary should review & approve the initial validation report
how should an actuary evalulate a model outside actuary’s expertise
make a reasonable attempt at understanding the model’s
- specifications
- validation (extent to which experts were involved)
- risk-rating
- complexity
- control framework
give an example of a model outside actuary’s expertise
a credit-scoring model
what is the purpose of sensitivity testing regarding models
- to validate a model
- to understand the relationship bewteen inputs/outputs
- to develop a sense of comfort with the model
how can model assumptions be tested in the context of sensitivity-testing
- test assumptions outside the expected range
- test assumptions singly and then in combination
- test assumptions with a nonlinear relationship between inputs/outputs
what types of validation should be done when using a model
DAR: validation of data, assumptions, results
- data should be ‘RelSuff’
- validate non-global assumptions that vary by model run
- results should be ‘reasonable’ relative to input
what does it mean for data to be reliable
- data reconciles to audited sources
- data is reasonable with respect to prior period data
what does it mean for data to be sufficient
- data fits model specification
- data is available in a consistent format
what checks can be done to validate the results of a model
- inputs/outputs should be consistent (input data should match similar fields in output file)
- results should be reasonable in both magnitude & direction (small change in inputs causes a small change in outputs)
compare uni-dimensional model risk-rating with two-dimensional rating
uni-dimensional approach:
- rating from 1-20
- based on financial significance, complexity, expertise of users, docs
two-dimensional approach:
- assessed separately for severity & likelihood of failure
- final rating is a balance of these
