Part 2

Question 1

Laplace’s Demon

Answer 1

An intelligence which at a given instant knew all the forces acting in nature and the position of every object in the universe - if endowed with a brain sufficiently vast to make all necessary calculations - could describe with a single formula the motions of the largest astronomical bodies and those of the smallest atoms. To such an intelligence, nothing would be uncertain; the future, like the past, would be an open book.

Question 2

Uncertainty

Answer 2

Rather than thinking of what we do not know o model we need to think of what we know and how certain these things are.

We need to quantify what we know our degree of belief.

Question 3

Probabilities

Answer 3

• Frequentist

- Bayesian

Question 4

Frequentist

Answer 4

(from Part 1)

A probability is the frequency of a repeatable event.

Question 5

Bayesian

Answer 5

(new)

A probability quantifies a degree of uncertainty in a statement.

Question 6

Bayes Rule

Answer 6

p(B|A) = p(A|B)p(B) / p(A)

Question 7

Bayesian Method

Answer 7

• Considers the data that I observe.
• Requires a prior assumption.
• A quantification of what I know before I see the data.
• Updates prior knowledge with data.
• The prior encodes an assumption.

Question 8

Frequentist Method

Answer 8

• We look at distributions of different data sets.
• Requires infinite amounts of data.
• Parameters are fixed.

Question 9

No Free Lunch Theorem

Answer 9

We have dubbed the associated results NFL theorems because they demonstrate that if an algorithm performs well on a certain class of problems then it necessarily pays for that with degraded performance on the set of all remaining problems.
Wolpert, D. H., & Macready, W. G. (1997).