Lecture 5

Question 1

What is model comparison?

Answer 1

A process that balances simplicity and accuracy when evaluating multiple models.

Question 2

What is the Akaike Information Criterion (AIC)?

Answer 2

A measure of model quality that penalizes complexity.

Question 3

What are Akaike weights and evidence ratios?

Answer 3

• Akaike weights: Probabilities that a model is the best among tested models.
• Evidence ratios: Quantify relative support for different models by comparing their Akaike weights.

Question 4

Why is it important to use the same dataset for model comparison?

Answer 4

To ensure fairness and consistency in evaluations.

Question 5

What does a lower AIC value indicate?

Answer 5

That the model has a better balance of fit and simplicity compared to models with higher AIC values.

Question 6

What is AICc, and why is it used?

Answer 6

A corrected version of AIC that adjusts for small sample sizes to reduce bias.

Question 9

What is the trade-off involved in model comparison?

Answer 9

More complex models may fit data better but risk overfitting, while simpler models may generalize better but fail to capture nuances.

Question 10

What is the goal of model comparison?

Answer 10

To identify the best model by balancing goodness of fit and model complexity to avoid overfitting.

Question 11

What is Akaike Information Criterion (AIC)?

Answer 11

AIC is a statistical measure that evaluates the goodness of fit of a model while penalizing for the number of parameters. The lower the AIC, the better the model.

Question 12

What is the formula for AIC?

Answer 12

AIC = -2L + 2k, where L is the likelihood of the model parameters given the observations, and k is the number of fitted parameters.

Question 13

Why is AICc recommended for small sample sizes?

Answer 13

AICc includes a correction term (2k(k+1)/(n-k-1)) that reduces bias in small sample sizes, making the AIC more accurate.

Question 14

What are delta AICc values?

Answer 14

Delta AICc values represent the difference between the AICc of each model and the best model. They are used to rank models.

Question 15

What are Akaike weights?

Answer 15

Akaike weights are probabilities that indicate how likely a model is to be the best among those compared, based on the observations.

Question 16

How are evidence ratios calculated?

Answer 16

Evidence ratios compare the likelihood of one model versus another by taking the ratio of their Akaike weights.

Question 17

What is k-fold cross-validation?

Answer 17

A technique that divides the dataset into k parts and validates the model on one part while calibrating on the others, repeating this process k times and averaging the errors.

Question 18

What are some challenges with k-fold cross-validation?

Answer 18

Issues include choosing the appropriate value for k, increased computation time, and potential dependencies between subsets.

Question 19

What does Occam’s razor suggest in model comparison?

Answer 19

It suggests preferring simpler models when they have similar goodness of fit, as they make fewer assumptions.

Question 20

Why must all models be fitted on the same dataset for valid AIC comparison?

Answer 20

To ensure consistency in comparisons, since differences in datasets could lead to biased evaluations.