Measures Flashcards

1
Q

Brier Score

A
  • measures accuracy of predicition models
  • mean square difference between predicted probability and actual outcomes
  • 0 indicates perfect predictions
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

ECE

A

Expected Calibration Error
* measures how well model’s predicted probabilities reflect the true likelihood
* partions into bins then weighted average across all bins

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

MAE

A

Mean Absolute Error
* measure of accuracy of predictions
* measure absolute differences between predicted values and actual values
* low MAE - better predictions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

RMSE

A

Root Mean Square Error
* measure accuracy of model
* calculates square root of average square differences between predicted and actual values
* low RMSE - better predictions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

out-of-sample R^2

A
  • compare predictive performance of two models by assessing how much one model improves upon the other in terms of MSE
  • > 0 - model 1 better
  • < 0 - model 2 better
  • = 0 - same MSE
How well did you know this?
1
Not at all
2
3
4
5
Perfectly