Empirical Risk Minimization Flashcards
1
Q
What is the hypothesis of a model?
A
It’s the function used to make prediction.
2
Q
What is the True risk associated to an hypothesis?
A
R(h) = E[ L(y,h(x)) ]
3
Q
What is the optimal hypothesis?
A
h* = argmin R(h)
4
Q
What is the Empirical risk of an hypothesis?
A
It’s the risk calculated on the training set:
R_emp(h) = 1/N * Σ L(y(i), h(x(i)))
5
Q
What is bias-variance decomposition?
A
Expected error of an hypothesis is
bias² + variance + noise
6
Q
What is irreductible error?
A
It’s the part of the expected error that is independent of h(x).
7
Q
What is Occam’s Razor?
A
“One should not increase, beyond what is necessary, the number of entities required to explain anything.” (e.g. “Seek the simplest explanation.”)
