Week 1: Probability Space, Conditional, Independence

Question 1

What is Statistics?

Answer 1

Statistics uses observed data to make inferences about the process that generated the data. It can also be used to determine how to control the uncertainty in results.

Question 2

What is Frequentist Inference?

Answer 2

Frequentist Inference assumes the probability is interpreted as an approximate mean observed when running some random experiment a large number (N) times.

Question 3

What is Bayesian Inference?

Answer 3

Bayesian Inference bases probability on a subjective degree of belief. The higher the probability of an event, the more likely the event is to happen.

Question 4

Probability Space

Answer 4

Structured framework that allows us to consistently measure uncertainty. Probability space is composed of sample space, sigma-algebra, probability measure.

Question 5

Sample Space

Answer 5

Set off all possible outcomes of an experiment. (Ex: for die toss it is {1,2,3,4,5,6})

Question 6

Sigma-Algebra

Answer 6

Collection of events (subsets) of sample space. This is the things we choose to care about from the sample space (ex: if we only want to measure the probability of getting a number less than 3 on a die toss, it is {1,2})

Question 7

Probability Measure

Answer 7

Assigns probabilities to each event. Ex: saying that there is a 1/6 chance of each specific die face showing up, or a 1/2 chance of an even number appearing

Question 8

Mutually Exclusive

Answer 8

Two events cannot occur simultaneously. They are incompatible.

Question 9

Probability of a subset (Sigma-Algebra)

Answer 9

P(A) = Sigma-Algebra Size / Sample Space Size

Example: in a dice roll, the normal probability is 1/6 (when sigma-algebra is {i}), but if I do subset of {1,3,5}, the probability becomes 1/2 (3/6).

Question 10

Theorem

Answer 10

Proven statement or proposition that is widely accepted based on deductive reasoning. It often represents a significant result in mathematics or statistics.

Question 11

Corollary

Answer 11

A proposition that follows easily from from a theorem with minimal additional proof. Typically extends the theorem or applies it to a particular case

Question 12

P(∅) = 0

Answer 12

The probability of the empty set (which means nothing happens) is always zero because it is impossible for an event that includes no outcomes to occur

Question 13

If A ⊂ B, then P(B \ A) = P(B) − P(A):

Answer 13

If event A is a subset of event B, the probability of B happening without A happening is the probability of B minus A

Question 14

If A ⊂ B, then P(A) ≤ P(B):

Answer 14

If event A is a part of event B, the probability of A is less than or equal to the probability of B. A can’t be more likely than B if it needs B to happen in order to occur.

Question 15

P(A^c) = 1 − P(A), where A^c := Ω \ A:

Answer 15

The probability of the complement of event A (meaning A does not happen) is equal to 1-P(A)

Question 16

P(A ∪ B) = P(A) + P(B) − P(A ∩ B):

Answer 16

The probability of either A or B happening, or both, is the sum of the probabilities of A and B minus the probability of both A and B happening to avoid the overlap (double counting)

Question 17

P(A|B) = P(A ∩ B) / P(B)

Answer 17

Probability of event A happening given that B has already happened is equal to the probability of both A and B happening (intersection) divided by the probability of B happening

Question 18

P(A ∩ B) = P(A)P(B).

Answer 18

This is only true if A and B are independent events. If A and B are independent, the probability that both A and B occur (intersection) is the product of their individual probabilities. These two events have no relation to each other. One occurring has zero impact on the probability that the other occurs.

Question 19

What is the probability of getting Heads or Tails three times in a row?

Answer 19

Sigma-Alpha Size (subset) / Sample Space Size (possible outcomes) for each flip

3 flips probability prediction = subset size / ((possible outcomes)^3)

{(H,H,H), (T,T,T)} = 2 / (2^3) = 1/4

Question 20

Bayes’ Theorem

Answer 20

Provides an update to the probability of event A after learning that event B has occurred. New A becomes P(A|B) = P(B|A)P(A) / P(B)

Question 21

Law of Total Probability Theorem

Answer 21

To find the probability of an event 1, we can break down different scenarios for event 2 (to which event 1 is dependent), calculate the conditional probability of event 1 being X (rather than Y) given each scenario of event 2, and then combine these probabilities by weighting them with the probability of each scenario happening.

Outcomes in event 2 are mutually exclusive and collectively exhaustive (equal 1 in sum / fill the sample space).

P(X)=P(X∣A)P(A)+P(X∣B)P(B), where A and B are outcomes of event 2