Ch.8 Backtesting and Stresstesting

Question 1

What is backtesting? How do we use it to test the effectiveness of a VaR Forecast?

Answer 1

Backtesting tests in-sample forecasts of different risk models using history. In the context of VaR, we analyze its accuracy using violations.

Question 2

What is the difference between an Estimation Window and a Testing Window.

Answer 2

Estimation window: the sub-sample used to fit a forecasting model.

Testing Window: The sub-sample on which we forecast risk.

Question 3

What is the Violation Ratio?

Answer 3

The ratio of observed number of violations over the expected number of violations.

Question 4

What are the different data requirements for the estimation window of the following risk models: EWMA, HS, GARCH?

Answer 4

EWMA: 30
HS: 300 for VaR 1%
GARCH: 500+

Question 5

What is the Bernoulli Coverage Test?

Answer 5

A test to confirm or deny if the number of observed violations is equal to the expected number of violations.

Question 6

What is the main problem of independence tests?

Answer 6

They can only test of one type of dependence.

Question 7

What is one way to backtest expected shortfall?

Answer 7

On average, Expected shortfall should equal the average loss given a violation. Thus, on days with violations, divide the return by the ES. This number should equal 1 across violations.

Question 8

What are the 2 main problems with backtesting?

Answer 8

1. It assumes that the structure of the system at hand doesn’t change, which isn’t the case.
2. It can be easily manipulated, and tends to produce overfit the past.