probability and stats

Question 1

Explain P(A)P(B) in plain English

Answer 1

The probability of independent events A and B occurring.

Question 2

Explain P(A and B) in plain English

Answer 2

The probability of events A and B occurring together.

Question 3

And rule for independence:

P(A and B) = ?

Answer 3

P(A and B) = P(A)* P(B)

assumes A, B are independent events

Question 4

And rule for dependence

P(A and B) = ?

Answer 4

P(A and B) = P(A) * P(B | A)
P(A and B) = P(B) * P(A | B)

assumes A, B are dependent events

Question 5

Are independent events are mutually exclusive?

Answer 5

No! Common fallacy. Independent events are NOT mutually exclusive!

Question 6

Or rule (generalized to mutually exclusive and non-mutually exclusive events):
P(a or b) =

Answer 6

P(a) + P(b) - P(a and b)

Question 7

Or rule for mutually exclusive events:

P(a or b) = ?

Answer 7

P(a or b) = P(a) + P(b)

Question 8

What’s the definition of conditional probability?

Answer 8

P(A | B) = P(A and B) / P(B)

probability of A given that B occurred
(only valid when P(B) > 0)

P(B) is the total outcome space. You know B happened, so you’re in that space, hence it’s the denominator.

Question 9

P(A | B) = 0.2

What is P(~A | B)?

Answer 9

P(~A | B) = 0.8

Realize that you’re summing over outcomes for A, not B!

Question 10

Conditionalized version of Bayes theorem in the context of general background evidence E:

P(X | Y, E) = ?

Answer 10

P(X | Y, E) = P(X | E) * P(Y | X, E) / P(Y | E)

Question 11

Conditionalized version of marginalization in the context of general background evidence E:

P(X | E) = ?

Answer 11

P(X | E) = sum over y of P(X, Y = y | E)

Question 12

What does the following statement mean in plain english?

P(X | Y, E) = P(X|E)

Answer 12

X is conditionally independent of Y given E

Question 13

What does the following statement mean in plain english?

P(Y | X, E) = P(Y | E)

Answer 13

X is conditionally independent of Y given E

Question 14

What does the following statement mean in plain english?

P(X, Y | E) = P(X | E) P(Y | E)

Answer 14

X is conditionally independent of Y given E

Question 15

Marginal independence: produce 2 other equivalent statements that imply each other:

P(X|Y) = P(X)
…
…

Answer 15

P(X|Y) = P(X)
P(Y|X) = P(Y)
P(X, Y) = P(X) P(Y)

All imply each other.

Question 16

Conditional independence: produce 2 other equivalent statements that imply each other:

P(X|Y, E) = P(X | E)
…
…

Answer 16

P(X|Y, E) = P(X | E)
P(Y | X, E) = P(Y | E)
P(X, Y | E) = P(X | E) P(Y | E)

Question 17

what’s a synonym for disjoint?

Answer 17

mutually exclusive

Question 18

what’s a synonym for mutually exclusive?

Answer 18

disjoint

Question 19

what does it mean when A is disjoint from B?

Answer 19

A and B cannot happen at the same time

Question 20

How do you determine if events A and B are mutually exclusive?

Answer 20

P(A and B) = 0

Question 21

Can you have a events A, B that are independent and disjoint of each other?

Answer 21

If at least one of the events has zero probability, then the two events can be mutually exclusive and independent simultaneously. Let A be the empty set, for example, and let B be any event. Then they are mutually exclusive (because their intersection is empty) and they are independent (because the probability of their intersection is equal to the product of their individual probabilities, which is zero).

However, if both events have non-zero probability, then they cannot be mutually exclusive and independent simultaneously. “Mutually exclusive” implies that the intersection of the two events has zero probability, but the events themselves have non-zero probabilities, so the product of their probabilities cannot be zero.

Question 22

Can you have a events A, B that are independent and NOT disjoint of each other?

Answer 22

Yes, of course

Question 23

Apply conditionalized bayes rule (flip C and D):

P(C|A,B,D) = …

Answer 23

Background evidence = A, B

P(C | A,B,D) = P(D | C,A,B) * P(C | A, B) / P(D | A, B)

Question 24

What is a prior probability?

Answer 24

a probability w/o conditions

e.g. P(A)

Question 25

What is a posterior probability?

Answer 25

a probability w/ conditions

e.g. P(A | B)

Question 26

(B) —–> (A)

Which node is parent and which is child?

Answer 26

B causes A. A is the child; B is the parent.

(parent) —-> (child)

A parent ‘causes’ the child

Question 27

Does independence imply conditional independence? Are they the same concept?

Answer 27

No.

https://math.stackexchange.com/questions/22407/independence-and-conditional-independence-between-random-variables

Question 28

Define Unigram model Pu(S)

where S is a sequence of letters

Answer 28

Pu(S) = Product from l=1 to L of P1(character in sequence at l)

Question 29

Define bigram model Pb(S)

where S is a sequence of letters

Answer 29

Pb(S) = Pu(character at l=1) * (Product from l=2 to L of P(l | l-1))

Question 30

How do you calculate bigram probability P(l | l-1) aka P(letter | prev_letter) ?

Answer 30

= count(letter l-1 followed by l) / count(letter l-1 followed by any letter)

Question 31

S = “aabbaa”

Calculate bigram probability P(letter = a | prev letter = a)

Answer 31

Since S = “aabbaa”,

P(letter = a | prev letter = a) = count(a followed by a) / count(a followed by any letter) = 2/3

Notice the denominator is 3, not 4 even though there are 4 occurrences of a. The reason is because the ‘a’ in the last index of the sequence S is not followed by anything. We don’t count it.

In HW4 bigram programming example, Saul processed the dataset such that we had an end-of-sentence terminator to simplify our work for us (didn’t have to count for this effect).