Probability

Question 1

Define the population, a sample an event and a singleton.

Answer 1

population: set of all entities under study
sample: an element of the sample space: the set of all outcomes
event: subset of the sample space
singleton: an individual element of the sample space

Question 2

Define the probability measure and give the three axioms

Answer 2

P : \Omega \to [0, 1]
P(Omega) = 1
P(A U B) = P(A) + P(B) if A n B = 0

Question 3

For a finite sample space Ω, suppose that each singleton is equally likely. What is the probability of an event A?

Answer 3

A / # Omega

Question 4

Compare the probability of a union of two events A and B to the individual probabilities for those events individually. What about intersections?

Answer 4

P(A U B) >= P(A) or P(B)

P(A n B) <= P(A) or P(B)

Question 5

Give the formula for the probability of event A conditional on event B.

Answer 5

P(A|B) = P(A n B)/P(B)

Question 6

State Bayes’ theorem. What is this useful for?

Answer 6

P(A|B) = P(B|A)P(A)/P(B)

This is useful for reversing conditionals

Question 7

Define independence.

Answer 7

P(A|B) = P(A) i.e. P(A n B) = P(A)P(B)

Question 8

What is a random variable?

Answer 8

A map X : \Omega -> IR subject to randomness

Question 9

Give an example of a random variable and a variable that is not random.

Answer 9

Result of a dice throw.

The number of players on a football pitch

Question 10

What is the probability a random variable X is in some subset S of the real numbers?

Answer 10

P(X \in S) = P({\omega \in Omega : X(\omega) \in S}

Question 11

Give an example of a finite and an infinite discrete random variable.

Answer 11

Result of a dice throw.

random choice of integer

Question 12

Define the CDF

Answer 12

F_X(z) = P(X <= z)

Question 13

Define the PMF. What type of random variable is this used for?

Answer 13

f_X(z) = P(X = z)

This is for discrete random variables

Question 14

Define the PDF. What type of random variable is this used for?

Answer 14

P(a <= z <= b) = \int_a^b dz f_X (z)

Question 15

Give some examples of normally distributed random variables.

Answer 15

heights of females

test scores

Question 16

Give some examples of Poisson distributed random variables.

Answer 16

goals in football matches

number of bones a person has broken

Question 17

Define the joint PMF/PDF for two random variables X and Y.

Answer 17

P(X \in [a,b], y \in [c,d]) = \int_a^b \int_c^d dz1 dz2 f_(X, Y) (z1, z2)

Question 18

How does this simplify if X and Y are independent?

Answer 18

f_(X,Y)(z1,z2) = f_X(z1)f_Y(z2)

Question 19

What is the PMF/PDF for a random variable X conditional on another random variable Y?

Answer 19

P(X \in [a,b] | Y \in [c,d]) = \int a^b dz f_(X|Y \in [c,d])

Question 20

Define the expectation of a discrete and a continuous random variable.

Answer 20

E[X] = \sum_x x f_X(x)
E[X} = \int_-infty^infty x f_X (x)

Question 21

What is the expectation of a function of a random variable?

Answer 21

E[g(X)] = \int_(-\infty)^in\fty g(x) f_X(x)

Question 22

What is the variance of a random variable and what does it measure?

Answer 22

var(X) = E[(X-E[X])^2]

measures expected squared deviation from the expectation

Question 23

What is the covariance between two random variables X and Y? What does it measure?

Answer 23

cov(X,Y) = E[(X-E[X])(Y-E[Y])]

measures the extent to which the two variables are linearly related

Question 24

Why can the covariance sometimes be hard to interpret.

Answer 24

It is dependent on the scale of X and Y

Question 25

Define the correlation between two random variables X and Y.

Answer 25

corr(X,Y) = cov(X,Y) / \sqrt(var(X)var(Y))

Question 26

Define the conditional expectation for a random variable X conditional on Y in the case that they are discrete and continuous.

Answer 26

E[X|Y] = \int_(-\infty)^(\infty) dx x f_(X|Y) (x)

Question 27

State the law of large numbers and the central limit theorem

Answer 27

Given a large enough sample size, the sample mean tends toward the population expectation.
CLT states that, in the limit N\to\infty, the sample mean is normally distributed, centred at the expectation and with vanishing variance