Chapter 8 Flashcards
How do you analyse model adequacy?
Model adequacy is measured by comparing model outputs with measured data and analysing the deviation between the two.
What is validation?
Validation refers to the quantification of our belief in the predictive capability of the model through comparison with experimental data.
What is better to use instead of “validation”?
Model confidence, performance or evaluation.
What does the level of model adequacy convey?
It indicates the level of precision and accuracy of the model.
What methods can we use to compare experimental data with model output?
- intuitive, graphical approach
- quantitative statistical approach
What is the statistical approach to compare outputs and data?
Quantifying model deviation:
Difference between modelled and measured value (residual/error term)
What is a bias?
Bias is a systematic directional difference of the prediction from the observation.
What is an ideal situation in terms of bias and repeatability?
Low bias (deviations distributed around a mean of 0) & high repeatability (variance of model predictions is low)
What is a criteria that we can use to statistically compare outputs and data?
Root mean square error or Normalised RMSE
What is the equation for RMSE?
RMSE:
sqrt(sum((Predictions - Observations)^2)/N)
What is the equation for NRMSE?
NRMSE:
RMSE / avg(Observations)
What is accuracy?
Accuracy measures how closely model-predicted values are to the true values.
What is precision?
Precision measures how closely individual model-predicted values are within each other.
What do inaccuracy and imprecision mean?
Inaccuracy (bias) is the systematic deviation from the truth.
Imprecision (uncertainty) refers to the magnitude of the scatter about the average mean.
How and why can a database be split for model calibration and testing?
The data used for calibration and testing should be independent otherwise errors or biases from the model can result.
The database can be split in two for the two different purposes. It can either be split by time (if subject remains the same) or by space (if they are believed to behave the same way but are still independent from each other).
