Fundamentals of Probability

Question 1

Three steps to generate a statistical model

Answer 1

(1) What is the data-generating process (DGP)?
(2) Build an appropriate probability model that reflects the assumed DGP including
assumptions of how Y is distributed (i.e., stochastic component)
(3) Come-up with a parameterization of the stuff that gets estimated (i.e., systematic component) and theory of inference to derive statistical model

Question 2

Data-generating process

Answer 2

This is the joint probability distribution that is supposed to characterize the entire population from which the data set has been drawn.

Question 3

Stochastic Component

Answer 3

The assumption about the way Y is distributed, in the case of linear regression it is an assumption about the normal distribution
yi~N(yi|μi, σ^2)

Question 4

Systematic Component

Answer 4

Parameterization of the stuff that gets estimated
μi=B0+B1Xi+B2X2+….

Question 5

Population regression function

Answer 5

yi=alpha+betax1+ui(error term), i=1,…,n

Question 6

Sample regression function

Answer 6

yi_hat=(alpha_hat)+(beta_hat)xi, i=1,…,n

Question 7

Experiment

Answer 7

Repeatable procedure for making an observation

Question 8

Outcome

Answer 8

possible result of repeatable procedure for making observation

Question 9

The sample space (Ω) of an experiment

Answer 9

Set of all possible outcomes

Question 10

An event

Answer 10

Subset of the sample space, i.e., any set of outcomes

Question 11

The probability of an event

Answer 11

it’s long-run relative frequency.

Question 12

A ∪ B
Give the operation name, definition and interpretation

Answer 12

Union
elements either in A or B or in both occur
either A or B or both

Question 13

A ∩ B
Give the operation name, definition and interpretation

Answer 13

Intersection
elements both in A and B
both A and B occur

Question 14

A_hat
Give the operation name, definition and interpretation

Answer 14

Complement
elements not in A
A does not occur

Question 15

A ⊆ B

Answer 15

• If B contains A
  : “when A occurs, so does B (but not
  necessarily vice versa)”

Question 16

The intuitive definition of probability

Answer 16

assigning real numbers to every element of the
sample space in a way that the sum of all such numbers is 1.

Question 17

• A random variable

Answer 17

function that assigns a number to each outcome of the
sample space of an experiment.

Question 18

Probability distributions

Answer 18

for all possible
outcomes, it tells us the probabilities for these outcomes to occur.

Question 19

Which types of distributions exist?

Answer 19

Discrete, Continuous

Question 20

What is the discrete distribution?

Answer 20

e.g., Bernoulli, Binomial, Poisson

Question 21

What are the Continuous distributions?

Answer 21

e.g., Uniform, Normal, Logistic, t-distribution

Question 22

Probability density function: definition and formal way to write it

Answer 22

Distribution of probabilities for all values of a random variable X
The probabilities p(x) or P(X = x) for all values of a random variable X form the probability density function (PDF).

Probability density function (PDF): What is the probability that we get
* exactly xi
(for discrete distributions)?
* a ≤ xi ≤ b (for continuous distributions)?

Question 23

Cumulative distribution function (CDF):

Answer 23

The probability of observing a value less or equal than x

cumulative distribution function (CDF): What is the probability that we get some
value equal to or smaller than xi?

Question 24

Expected Value: definition and formula

Answer 24

Specifies the center of the probability distribution
X discrete:
E(X) = ∑(all x) * xp(x)
X continuous:
E(X) = ∫( +∞ −∞) * xp(x)dx

Question 25

The variance of the probability distribution: definition and formula

Answer 25

Specifies the spread of the probability distribution
X discrete: Var(X) = ∑(all x) * (x − E(X))^2 * p(x)

X continuous: Var(X) = ∫ (+∞ −∞) * (x − E(X))2 * p(x)d

Question 26

Binomial Distribution

Answer 26

Distribution of a binomial random variable K that represents the number of
‘successes’ in n outcomes of a binomial process

Question 27

A binomial process is given by:

Answer 27

1. n independent Bernoulli trials
2. Only two possible outcomes, which are arbitrarily called ‘success’ and ‘failure
3. Failure and success probabilities assumed to remain constant over trials

Question 28

Mean and variance of the binominal distribution: definition and formula

Answer 28

Let n be the number of trials, p the probability of success
The Binomial distribution has mean (expected value):
E(K) = np
and variance
Var(K) = np(1 − p)

Question 29

The binomial probability mass function formula

Answer 29

f(k; n, p) = P(K = k) = (^n k) p^k(1 − p)^(n−k)

Question 30

The binomial cumulative distribution function formula

Answer 30

F(k; n, p) = P(K ≤ k)=∑(k above, i=0 below)(^n i)p^1*(1-p)^(n-i)

Question 31

Normal distribution: formal expression

Answer 31

Normal distribution N (µ, σ2)

Question 32

Normal distribution: description

Answer 32

Continuous distribution that describes data clustered around the mean.
* Uniquely determined by its mean/median/mode µ and variance σ^2
* Importance of the normal distribution because of the Central Limit Theorem.

Question 33

The formula of the normal distribution: Probability Density Function (with two parameters)

Answer 33

f(x; µ, σ2) = 1/(√2πσ^2)*exp *[−(x − µ)^2 / 2σ^2]

Question 34

The formula of the normal distribution: Cumulative distribution function

Answer 34

F(x; µ, σ^2) =∫ (^x −∞)f(t; µ, σ^2)dt= Φ(x-µ/σ)

Question 35

For what z-score is used? Give formula:

Answer 35

• To compare variables from different distributions, we can standardize them by
  building so called z-scores
  zi =(xi − (x_hat))/( σ)

Question 36

What are the variance and the mean of standard normal distribution?

Answer 36

squared variance=1, mean=0

Question 37

Central Limit Theorem

Answer 37

1.We have a population distribution (any, not necessarily normal) with mean µ and
variance σ^2 and we are interested in its mean.
2.Repeatedly taking samples from that population and calculating the mean for each
sample yields the sampling distribution of the mean
3.This sampling distribution approaches a normal distribution with mean µ and
variance σ 2/n as n increases.