Bayesian Decision Theory

Question 1

Bayes Theorem

Answer 1

posterior = likelihood * prior / evidence
P(w/x) = P(x/w)*P(w)/P(x)

Question 2

Classification Error

Answer 2

P(error|x) = P(w1|x) if we decide w2, P(w2|x) if we decide w1
Error rate = IN[P(error|x)P(x)dx]

Question 3

Optimal Classifiers

Answer 3

Bayes decision rule: w1 if P(w1|x)>P(w2|x), w2 else

Bayes error rate: R = IN[min(P(w1|x), P(w2|x)P(x)dx]

Question 4

How would you estimate the error if there was no upper bound that are both tight and analytically integrable. 1 low-dim 2 high-dim

Answer 4

1 numerical integration

2 approximate integral as sum