Chapter 3: Fundamentals of Statistics Flashcards
What is a random variable?
This is one that takes on numerical values and has an outcome that is determined by an experiment.
In other words, a random variable is defined as a variable that takes an obsered random (and not deterministic) value.
For this chapter we denote random variables by uppercase letters (usually W, X, Y, and Z), whereas outcomes of random variables are denoted by the corresponding lowercase letters (w, x, y, and z).
What is an example of a random variable?
▶ Outcome of a fair coin.
▶ Outcome of a fair die
We are interested in the number of Tail when tossing a fair coin twice.
▶ The sample space of the 2 coin toss outcomes are:
▶ Ω = {HH, TH, HT , TT}
▶ We define X to be the number Head among the two tosses. Then X can take on 3 possible values:
X = 0 if HH
X = 1 if TH ∪ HT
X = 2 if TT
▶ The X’s sample space is: ΩX = {0, 1, 2}
What is a bernoulli random variable?
This is a random variable that can only take on the balues zero and one.
Bernoulli random variable is sometimes labelled as binary random variable.
What is a discrete random variable?
This is one that takes only a finite number of values.
A bernoulli random variable is the simplest example of a discrete random variable.
Other examples:
▶ number of tails when tossing a coin twice,
▶ number of students registering for a class.
What is an exampe of a bernoulli random variable?
The coin flipping example.
If one coin is ‘fair’ , then P(X = 1) = 1/2 (read as ‘the probability of X equals one is one-half).
Because probabilities must sum to one P(X = 0) = 1/2 also.
Whats symbol usually represents an unknown probability?
θ (theta)
e.g. the probability of any particular customer showing up can be any number between zero and one:
P(X = 1) = θ
P(X = 0) = 1 - θ
If θ = 0.75 then there is a 75% chance that a customer shows up afer making a reservation and a 25% chance that the customer does not show up.
How is any discrete random variable usually depicted?
It’s usually depicted/described by listing its possible value and the associated probability that it takes on each value.
If X takaes on the k possible values {x1, …, xk}, then the probabilities p1, p2, …, pk are defined by:
pj = P(X = xj), j = 1, 2, …, k
where each pj is between 0 and 1, and
p1 + p2 + … + pk = 1
What are probability density functions (pdf)?
The pdf of X summarises the information concerning the possible outcome of X and the corresponding probabilities:
f(xj) = pj, j = 1, 2, …, k
Suppose that X is the number of free throws made by a basketball player out of two attempts, so that X can take on the threes {0, 1, 2}.
Assume the pdf of X is given by:
f(0) = 0.2
f(1) = 0.44
f(2) = 0.36
What is the probability that the player makes at least one free throw?
At least one free throw = P(X ≥ 1)
P(X ≥ 1) = P(X = 1) + P(X = 2)
P(X ≥ 1) = 0.44 + 0.36
P(X ≥ 1) = 0.80
How do you draw a probability density function (pdf)?
f(x) = y axis
x = x axis
probabilies are straight vertical lines starting from 0 on the x axis and going to their repective limit on the y axis (shown on page 48 of textbook).
What are the types of random variables?
A discrete random variable takes on a finite number of values. For example:
▶ number of tail when tossing a coin twice,
▶ number of students registering for a class.
A continuous random variable takes on any value in a real interval. For example:
▶ Time to complete an assignment.
▶ Wages
▶ Return on stock market
What is a continuous random variable?
A variable X is a continuous random variable if it takes on any real value with zero probability.
The idea is that a continuous random variable can take on so many possible values that we cannot count them or match them up with the positive intergers.
For example:
▶ Time to complete an assignment.
▶ Wages
▶ Return on stock market
How are pdfs used for continuous random variables?
We use the pdf of a continuous random variable only to compute events involving a range of values, because it makes no sense to discuss probabiity that a continuous random variable takes on a particular value.
e.g. if a and b are constants where a <:b, the probability that X lies between the numbers a and b, (a ≤ X ≤ b), is the area under the pdf between points a and b.
To find this value you find the integral of the function f between points a and b.
How do you draw a pdf for continuous random variables?
f(x) = y axis)
x = x axis
area underneath the function (non linear line) represents the total probability, meaning the entire area under the pdf must always equal one (example on page 49 of textbook).
What are cumulative distribution functions (cdf) used for?
When computing probabilities for continuous random variables, it is easiest to work with the cdf.
If X is any random variable, then its cdf is defined for any real number x by:
F(x) = P(X ≤ x)
What are two important properties of cdfs that are useful for computing probabilities?
For any numbers c, P(X > c) = 1 - F(c)
For any numbers a < b, P(a < X ≤ b) = F(b) - F(a)
What are some useful (univerate) distributions?
Some discrete distributions that are useful in modeling social phenomena include the:
▶ Bernoulli, for binary outcomes (e.g. pass/fail in a test)
▶ Binomial, for independent repetitions of Bernoulli “trials” (e.g., number of successes in throwing a basketball)
▶ Poisson, for count variables (e.g. number of clicks on a site in a minute)
Some continuous distributions that are useful include the:
▶ Normal, for measurement errors
▶ Exponential, for waiting time for first occurrence (e.g., of first patient)
▶ Student t, Chi-square χ2, F distribution for testing hypothese
What is a joint distribution?
This analyses events with more than one random variable.
▶ We have these because we are usually interested in phenomena that involve more than one random variable:
▶ e.g wage and gender, temperature and covid infection, etc.
▶ Therefore we study joint distributions.
▶ The joint distribution is described by the joint CDF, or the joint PDF.
What is an example of joint distributions?
For example, the joint distribution of two discrete RVs:
▶ X : # of women among two customers
▶ Y : number of items bought
Is given by: (TABLE)
fXY X
_____________________________
| 0 1 2
{ 0 | 0.05 0.1 0.03
y{ 1 | 0.21 0.11 0.19
{ 2 | 0.08 0.15 0.08
▶ Each cell is the joint probability Pr(X = x ∩ Y = y ).
▶ For example, Pr(X = 0 ∩ Y = 0) = .05.
What are marginal distributions?
This gives the probabilities of various values of the variables in a subset without reference to the values of the other variables.
The marginal distributions of each RV can be obtained from the joint distribution:
fY (y ) = Pr (Y = y ) = ∑x Pr (Y = y ∩ X = x)
fX (x) = Pr (X = x) = ∑ y Pr (X = x ∩ Y = y )
Find the probability of Y = 0 when the marginal PDF of Y , is obtained by
computing the probabilities:
▶ Pr(Y = 0)
▶ Pr(Y = 1)
▶ Pr(Y = 2)
Pr (Y = 0) = ∑x=0,1,2 Pr (Y = 0 ∩ X = x)
(according to table)
When y = 0, X1= 0.05, X2 = 0.1 and X3 = 0.03.
Pr (Y = 0) = Pr ({Y = 0 ∩ X = 0}) + Pr ({Y = 0 ∩ X = 1})+ Pr ({Y = 0 ∩ X = 2})
= 0.05 + 0.1 + 0.03 = 0.18
When are random variables independent?
▶ Two RVs are independent when knowing the value of one does not change the distribution (the probabilities) of the other.
▶ Formally, two RVs are independent if and only if the joint distribution is the product of the marginal distributions:
fXY (x, y = fX (x) * fY (y )
▶ Independence is symmetric: If Y is independent of X then X is independent of Y.
How do you check the independence of random variables?
▶ To check independence, we need to check for all pairs (x, y ) if Pr (Y = y ∩ X = x) = Pr (Y = y) · Pr (X = x) .
▶ In our example independence clearly does not hold, since
e.g.
Pr (Y = 0 ∩ X = 1) = ̸ = Pr (Y = 0) · Pr (X = 1)
0.10 = ̸ = 0.18 ∗ 0.36
(according to table)
Pr (Y = 0 ∩ X = 1) = 0.1 (when y = 0 and x = 1)
Pr (Y = 0) = 0.18
Pr (X = 1) = 0.36
What are conditional distributions?
A conditional distribution is a distribution of values for one variable that exists when you specify the values of other variables. This type of distribution allows you to assess the dispersal of your variable of interest under specific conditions, hence the name.
This information is summaries by the conditional probability density function, defined by:
fXY (y|x) = fX,Y (x,y)/fx(x)
for all values of x such that fx(x) > 0
How are conditional distributions depicted with discrete random variables?
The condititonal probability density function [fXY (y|x) = fX,Y (x,y)/fx(x)] is most easily seen when X and Y are discrete, then:
fY|X (y|x) = P(Y = y|X = x)
The right-hand side is read as ‘the probability that Y = y given that X = x
How are conditional distributions depicted with continuous random variables?
When Y is continuous, fX|y (y|x) is not interpretable directly as a probability, for the reasons discussed earlier, but condititional probabilities are found by computing areas under the conditional pdf.
