Chapter 2- Probabilistic Models

Question 1

what is a random variable?

Answer 1

a numeric quantity whose values map to the possible outcomes of an experiment

Question 2

what is a sample space, or alphabet?

Answer 2

consists of all possible events

Question 3

what is an event?

Answer 3

any subset of values that X can take

Question 4

what is the first rule of probability theory?

Answer 4

probabilities add up to 1

Question 5

what is the difference between estimated and true probabilities?

Answer 5

an estimate comes from a sample- it is a sample estimate

Question 6

what is another name for the true probability?

Answer 6

a population parameters

Question 7

what is joint probability?

Answer 7

AND

Question 8

from conditional probabilities, p(x,y) = ?

Answer 8

p(x|y)p(y) and vice versa, p(y|x)p(x)

Question 9

what is the rule for independent events?

Answer 9

p(x,y) = p(x)p(y)

Question 10

how is bayes theorem derived?

Answer 10

p(x,y) = p(x|y)p(y)
and vice versa p(x,y) = p(y|x)p(x)
equate these

Question 11

what is another word for joint probability?

Answer 11

marginalisation

Question 12

what is the formula for joint probability, p(X=x)?

Answer 12

sum for each y: p(x|y)p(y)

Question 13

what is the conditional independence assumption? (in words)

Answer 13

the features are conditionally independent of each other, given the class value

Question 14

what is the conditional independence assumption p(x1,x2,x3|y) = ?

Answer 14

p(x1|y)p(x2|y)p(x3|y)

Question 15

what is the conditional independence assumption p(X|y) = ?

Answer 15

multiply for each x: p(x|y)

Question 16

what is the naive bayes model?

Answer 16

we solve problems by making the conditional independence assumption

Question 17

a bayesian network is an example of?

Answer 17

a directed acyclic graph

Question 18

when might over/underfitting occur in a bayesian network?

Answer 18

if we have more links, the more complicated the probability distribution and hence more data is needed

Question 19

what is one of the great advantages of bayesian networks?

Answer 19

you can very naturally encode human knowledge about a given problem

Question 20

what is the chain rule for probability: P(A n B) = ?, and P(A1 n A2 n A3 n A4) = ?

Answer 20

P(A n B) = p(B|A)P(A)
or for more
P(A1 n A2 n A3 n A4) = p(A4 | A1 n A2 n A3)
= P(A4 | A3 n A2 n A1) p(A3 | A2 n A1)p(A2 | A1) p(A1)

Question 21

what does the chain rule tell us how to compute? Events A and B

Answer 21

The chain rules tells us how to compute the probability of event A happening AND then event B occurring afterwards.

Question 22

what is a random experiment?

Answer 22

random experiment is any experiment in which the outcome is uncertain (not known or determined in advance).

Question 23

what is P(A or B) if

a) they are disjoint
b) they are joint

Answer 23

a) P(A) + P(B)

b) P(A) + P(B) - P(A and B)

Question 24

what is a probability mass function?

Answer 24

discrete random variables take on a finite number of values.
Each value is associated with a probability of it occurring. t
The collection of these probabilities is the probability mass function

Question 25

give the bernoulli distribution

Answer 25

P(X = 0) = 1 - p
p(X = 1) = p

Question 26

give the binomial distribution, P(X=k) =

Answer 26

P(X = k) = (nCk)(p^k)(1-p)^(n-k)

Question 27

when would we use the binominal distribution

Answer 27

if experiment is repeated n times, the probability that we will see k successes is given by this probability mass function

Question 28

give the geometric distribution

Answer 28

P(X=x) = (1-p)^x-1 (p)

Question 29

when would we use the geometric distribution

Answer 29

it is useful for modelling the first occurrence of an outcome after repeated identical trials

Question 30

give the poisson distribution

Answer 30

P(X=x) = { lambda^x e(-lambda) } / x!

Question 31

when would we use the poisson distribution

Answer 31

if we had a rate e.g. 5 times a year

Question 32

if a discrete r.v. X has a pmf f(X) what is the expected value E[g(x)]

Answer 32

sum i: g(Xi)f(Xi)

Question 33

if a discrete r.v. X has a pmf f(X) what is the variance V[g(x)]

Answer 33

E[(g(X) - E(g(X)))^2]

E[g(X)^2] - E[g(X)]^2

Question 34

properties of Expectations

E[aX + b] =

Answer 34

aE[X] + b

Question 35

properties of variance:

V[aX+b] =

Answer 35

a^2V[X]

Question 36

what are the two schools of probability?

Answer 36

frequency based

belief based

Question 37

Describe the gamblers fallacy and how it demonstrates iid

Answer 37

A sequence of outcomes of spins of a fair or unfair roulette wheel is i.i.d

The outcome of the previous turn doesnt impact the next turn

The distribution of probabilities is the same each time i.e. 50/50 if not biased.

So, even if we’ve had 20 reds, there is still the same chance of having a black vs red next go

Question 38

Let the two events be the probabilities of persons A and B getting home in time for dinner, and the third event is the fact that a snow storm hit the city. If they live in different areas, are A and B conditionally independent given C?

Answer 38

Yes. That is, the knowledge that A is late does not tell you whether B will be late.

If they lived in the same neighbourhood they would not be.

Question 39

if two events are independent, are they also conditionally dependent?

Answer 39

not necessarily, it depends on the third event.

rolling a dice twice are two independent events. if the third event was the sum of them being 7, they would then be independent but not conditionally independent given c