Measures Flashcards
1
Q
Brier Score
A
- measures accuracy of predicition models
- mean square difference between predicted probability and actual outcomes
- 0 indicates perfect predictions
2
Q
ECE
A
Expected Calibration Error
* measures how well model’s predicted probabilities reflect the true likelihood
* partions into bins then weighted average across all bins
3
Q
MAE
A
Mean Absolute Error
* measure of accuracy of predictions
* measure absolute differences between predicted values and actual values
* low MAE - better predictions
4
Q
RMSE
A
Root Mean Square Error
* measure accuracy of model
* calculates square root of average square differences between predicted and actual values
* low RMSE - better predictions
5
Q
out-of-sample R^2
A
- compare predictive performance of two models by assessing how much one model improves upon the other in terms of MSE
- > 0 - model 1 better
- < 0 - model 2 better
- = 0 - same MSE