Week 8 - Decision theory

Question 1

Parameters vs hyper-parameters

Answer 1

Parameters= Parameters that define the distribution of the data

Hyper-parameters= Parameters that define the prior or posterior

Observe X ∼ Binomial(n, p), and use a Beta(α, β) prior, resulting
in a Beta(α˜, β˜) posterior.

Parameters: n, p (but n is fixed and known)
Hyper-parameters: α, β, α˜ , β˜

Question 2

Credibility factor (Z)

Answer 2

Measures how influential the data is on the posterior

Z ranges from 0 to 1
- High Z -> Data is more influential
- Low Z -> Prior distribution is more influential

Question 3

Decision theory

Answer 3

Seeks to determine optimal strategies for taking actions

Often used for deriving optimal estimators for Bayesian inference

More generally, it tells us how to use the posterior distribution

Question 4

Elements of Decision theory

Answer 4

θ denotes a state of nature
Θ is the set of all possible states of nature
Decision ‘a’ is called an action
‘A’ is the set of all possible actions
Require a loss function L(θ, a) - how good/bad an action is in the state of the world

Question 5

Loss function

Answer 5

Assigns a numerical value to the difference between predicted and actual outcomes, lower values indicate better performance as it states predictions are close to actual values

Squared error loss: L(θ, θˆ) = (θ − θˆ)^2

Absolute loss: L(θ, θˆ) = |θ − θˆ|

Question 6

Bayes estimator

Answer 6

The estimator that minimises the posterior expected
loss

Under the squared error loss -> the Bayes estimator is the posterior mean

Under the absolute loss -> the Bayes estimator is the posterior median