Fundamentals of Probability Flashcards
Three steps to generate a statistical model
(1) What is the data-generating process (DGP)?
(2) Build an appropriate probability model that reflects the assumed DGP including
assumptions of how Y is distributed (i.e., stochastic component)
(3) Come-up with a parameterization of the stuff that gets estimated (i.e., systematic component) and theory of inference to derive statistical model
Data-generating process
This is the joint probability distribution that is supposed to characterize the entire population from which the data set has been drawn.
Stochastic Component
The assumption about the way Y is distributed, in the case of linear regression it is an assumption about the normal distribution
yi~N(yi|μi, σ^2)
Systematic Component
Parameterization of the stuff that gets estimated
μi=B0+B1Xi+B2X2+….
Population regression function
yi=alpha+betax1+ui(error term), i=1,…,n
Sample regression function
yi_hat=(alpha_hat)+(beta_hat)xi, i=1,…,n
Experiment
Repeatable procedure for making an observation
Outcome
possible result of repeatable procedure for making observation
The sample space (Ω) of an experiment
Set of all possible outcomes
An event
Subset of the sample space, i.e., any set of outcomes
The probability of an event
it’s long-run relative frequency.
A ∪ B
Give the operation name, definition and interpretation
Union
elements either in A or B or in both occur
either A or B or both
A ∩ B
Give the operation name, definition and interpretation
Intersection
elements both in A and B
both A and B occur
A_hat
Give the operation name, definition and interpretation
Complement
elements not in A
A does not occur
A ⊆ B
- If B contains A
: “when A occurs, so does B (but not
necessarily vice versa)”
The intuitive definition of probability
assigning real numbers to every element of the
sample space in a way that the sum of all such numbers is 1.
- A random variable
function that assigns a number to each outcome of the
sample space of an experiment.
Probability distributions
for all possible
outcomes, it tells us the probabilities for these outcomes to occur.
Which types of distributions exist?
Discrete, Continuous
What is the discrete distribution?
e.g., Bernoulli, Binomial, Poisson
What are the Continuous distributions?
e.g., Uniform, Normal, Logistic, t-distribution
Probability density function: definition and formal way to write it
Distribution of probabilities for all values of a random variable X
The probabilities p(x) or P(X = x) for all values of a random variable X form the probability density function (PDF).
Probability density function (PDF): What is the probability that we get
* exactly xi
(for discrete distributions)?
* a ≤ xi ≤ b (for continuous distributions)?
Cumulative distribution function (CDF):
The probability of observing a value less or equal than x
cumulative distribution function (CDF): What is the probability that we get some
value equal to or smaller than xi?
Expected Value: definition and formula
Specifies the center of the probability distribution
X discrete:
E(X) = ∑(all x) * xp(x)
X continuous:
E(X) = ∫( +∞ −∞) * xp(x)dx
The variance of the probability distribution: definition and formula
Specifies the spread of the probability distribution
X discrete: Var(X) = ∑(all x) * (x − E(X))^2 * p(x)
X continuous: Var(X) = ∫ (+∞ −∞) * (x − E(X))2 * p(x)d
Binomial Distribution
Distribution of a binomial random variable K that represents the number of
‘successes’ in n outcomes of a binomial process
A binomial process is given by:
- n independent Bernoulli trials
- Only two possible outcomes, which are arbitrarily called ‘success’ and ‘failure
- Failure and success probabilities assumed to remain constant over trials
Mean and variance of the binominal distribution: definition and formula
Let n be the number of trials, p the probability of success
The Binomial distribution has mean (expected value):
E(K) = np
and variance
Var(K) = np(1 − p)
The binomial probability mass function formula
f(k; n, p) = P(K = k) = (^n k) p^k(1 − p)^(n−k)
The binomial cumulative distribution function formula
F(k; n, p) = P(K ≤ k)=∑(k above, i=0 below)(^n i)p^1*(1-p)^(n-i)
Normal distribution: formal expression
Normal distribution N (µ, σ2)
Normal distribution: description
Continuous distribution that describes data clustered around the mean.
* Uniquely determined by its mean/median/mode µ and variance σ^2
* Importance of the normal distribution because of the Central Limit Theorem.
The formula of the normal distribution: Probability Density Function (with two parameters)
f(x; µ, σ2) = 1/(√2πσ^2)*exp *[−(x − µ)^2 / 2σ^2]
The formula of the normal distribution: Cumulative distribution function
F(x; µ, σ^2) =∫ (^x −∞)f(t; µ, σ^2)dt= Φ(x-µ/σ)
For what z-score is used? Give formula:
- To compare variables from different distributions, we can standardize them by
building so called z-scores
zi =(xi − (x_hat))/( σ)
What are the variance and the mean of standard normal distribution?
squared variance=1, mean=0
Central Limit Theorem
1.We have a population distribution (any, not necessarily normal) with mean µ and
variance σ^2 and we are interested in its mean.
2.Repeatedly taking samples from that population and calculating the mean for each
sample yields the sampling distribution of the mean
3.This sampling distribution approaches a normal distribution with mean µ and
variance σ 2/n as n increases.
