Week 5 - Probability

Question 1

Axioms of probability

Answer 1

• Pr(A) >= 0, probability can’t be negative
• Adding all probabilities of sample space is 1
• If events are mutually exclusive (the events cannot happen simultaneously), their probabilities can be added up
  Pr(A or B) = Pr(A) + Pr(B)

Question 2

Addition rule

Answer 2

• If events overlap, then we subtract the overlap
  Pr(A or B) = Pr(A) + Pr(B) - Pr(A and B)

Question 3

Random variable

Answer 3

Something that can have different values, with each value representing a possible outcome of an event

Value of random variable that actually occurs is called a realisation of that random variable (no longer uppercase anymore)

Question 3

Probability distribution

Answer 3

Describes the probabilities of events for a given random variable

Question 3

Complementary events

Answer 3

If we can Pr(A), then ‘All outcomes not in A’
Pr(A) + Pr(A^c) = 1

Question 4

Random variable functions

Answer 4

Random variables can be discrete or continuous

• Any random variable X -> Cumulative distribution function (cdf)
• Discrete variables -> Probability mass function (pmf)
• Continuous variables -> Probability density function (pdf)

Question 5

Independence

Answer 5

Two events are independent if the occurrence of one does not change the probability of occurrence of the other

Question 6

Multiplication rule

Answer 6

A and B are independent events if:
Pr(A and B) = Pr(A) * Pr(B)

Question 7

Joint probability

Answer 7

Probability of outcomes for two or more variables

Question 8

Marginal probability

Answer 8

Probability of outcomes for a single variable

Question 9

Conditional probability

Answer 9

Probability of outcomes for a single variable given information about a second variable

Pr(A | B) = Pr(A and B) / Pr(B)

If independent then Pr(A | B) = Pr(A)

Question 10

Conditional probability distribution

Answer 10

Probability distribution where each probability is a conditional probability with the same condition

Question 11

General multiplication rule

Answer 11

Pr(A and B) = Pr(A | B) * Pr(B)

(vise-versa)

Question 12

Bayes’ theorem

Answer 12

Assuming we know Pr(B | A) but we want to know Pr(A | B)

Pr(A | B) = [Pr(B | A) * Pr(A)] / Pr(B)

Question 13

Expected value

Answer 13

Also know as mean

Denoted as E(X)

Question 14

Variance

Answer 14

Denoted as var(X)

Often σ^2 = var(X)

The standard deviation of X is given by sd(X) = σ =√σ^2

Question 15

Large of Large Numbers (LLN)

Answer 15

When a collection of random variables increases, the mean gets closer to µ (mean of the distribution that defines the variables)

Has to be iid