Chapter 3

Question 1

What is an experiment?

Answer 1

An experiment is a set of rules that governs a specific procedure, which can be indefinitely repeated and has a well-defined set of outcomes. An experiment with only one possible outcome is called a deterministic experiment, while an experiment with two or more possible outcomes is called a random experiment

Question 2

What is a trial?

Answer 2

A trial is any performance or exercise of a defined experiment.

Question 3

What is an outcome?

Answer 3

Result of a given trial.

Question 4

What is the sample space?

Answer 4

A set \omega of all possible outcomes of an experiment

Question 5

What does the classical definition of probability assume?

Answer 5

A discrete and uniform distribution of probabilities.

Question 6

What is a probability space?

Answer 6

Mathematical model of real-world processes.

Consists of:
- Sample space
- Set of Events
- Probabilities

Question 7

What are Kolmogorov’s Axioms?

Answer 7

1. The probability of an event \omega is non-negative
2. The probability that some event in the entire sample space will occur is 1
3. Sum of probabilities is homogeneous

Question 8

What is a stochastic variable?

Answer 8

A variable representing the outcome of a naturally real-valued random experiment

or, a a function X mapping the probability space of a random experiment to real numbers

Question 9

What is the stochastic process?

Answer 9

Stochastic processes are used to describe probabilities of non-deterministic systems.

Question 10

What is Ergodicity?

Answer 10

If all the statistics of a process 𝑋𝑘 may be determined from a single function 𝑋𝑖 𝜔0 of the process, it is said to be ergodic – that is, it behaves the same whether analyzed over time or averaged over space.

Question 11

What is the probability distribution function ?

Answer 11

The probability distribution function (pdf) (probability mass, probability density) describes the probability of a stochastic variable taking certain values.

Question 12

What is the probability mass function?

Answer 12

Probability distribution function for discrete probability distribution

Question 13

What is Probability density function?

Answer 13

Probability distribution function for Continuous probability distribution

Question 14

What is the cumulative distribution function?

Answer 14

The cumulative distribution function (cdf) describes the probability that a stochastic variable 𝑋 with a given probability distribution (discrete or continuous) will be found at a value less than or equal to 𝑥.

Question 15

What are Statistical Moments?

Answer 15

A moment can be seen as a quantitative measure of the shape of a set of points, i.e. the moments describe how a probability distribution is shaped

Question 16

What are the formulas for the k-th moment of a random variable X?

Answer 16

Continuous:

E[X^k] = \int_{-\infty}^{\infty} x^k p(x) dx

Discrete:

E[X^k] = \sum_{i=1}^N x_i^k p(x_i)

Question 17

What are the formulas for the k-th central moment of a random variable X?

Answer 17

Continuous:

E[(X-\mu)^k] = \int_{-\infty}^{\infty} (x-\mu)^k p(x) dx

Discrete:

E[(X-\mu)^k] = \sum_{i=1}^N (x_i-\mu)^k p(x_i)

Question 18

What are the formulas for the k-th standardized moment of a random variable X?

Answer 18

Continuous:

E[((X-\mu)/\sigma)^k] = (1/\sigma)^k * \int_{-\infty}^{\infty} (x-\mu)^k p(x) dx

Discrete:

E[((X-\mu)/\sigma)^k] = (1/\sigma)^k * \sum_{i=1}^N (x_i-\mu)^k p(x_i)

Question 19

What is expected value / expectation / mean? What are its formulas

Answer 19

1st moment of x

Discrete:
E[x] = \sum_{i=1}^N x_i p_d(x_i) = \micro

Continuous E[x] = \int_{-\infty}^{\infty} xp(x)dx = \micrp

Question 20

What is variance \sigma? What are its formulas?

Answer 20

Second statistical moment. Describes how far away the number lie from the mean.

Discrete: Var[x] = \sum_{i=1}^N (x_i - \micro)^2 * p_d(x_i) = E[(X-E[X])^2]

Continuous:
Var[x] = \int_{-\infty}^{\infty} (x-\micro)^2 p(x) dx = E[(X-E[X])^2]

Question 21

What is Skewdness? What are its formulas?

Answer 21

Normalized third central moment. Indicator of asymmetry

Discrete: S[x] = (1/\sigma^3) * \sum_{i=1}^N (x_i - \micro)^3 * p_d(x_i)

Continuous:
S[x] = (1/\sigma^3) * \int_{-\infty}^{\infty} (x-\micro)^3 p(x) dx

Question 22

What is Kurtosis. What are its formulas?

Answer 22

Fourth normalized moment. Degree of peakedness

Discrete: K[x] = (1/\sigma^4) * \sum_{i=1}^N (x_i - \micro)^4 * p_d(x_i)

Continuous:
K[x] = (1/\sigma^4) * \int_{-\infty}^{\infty} (x-\micro)^4 p(x) dx

Question 23

What is covariance?

Answer 23

The covariance (or cross covariance) of two stochastic variables 𝑋 and 𝑌 is a measure of how the variables change together.

Question 24

What is the formula for the correlation coefficient?

Answer 24

\rho = \frac{Cov(x,y)}{\sigma_x \sigma_y}

Question 25

What is the formula for covariance?

Answer 25

Cov(X,Y) = E[(X - E[X])(Y-E[Y])]

Question 26

What is conditional probability?

Answer 26

The conditional probability is the probability of an event under certain circumstances, more precisely when the sample space is limited to another event.

Question 27

What is the formula for conditional probability?

Answer 27

Let 𝐴 and 𝐵 be two events. Then the conditional probability of 𝐵 given that 𝐴 has occurred is given by :

P(B|A) = \frac{P(A N B)}{P(A)}

Question 28

What is Bayes’ Theorem?

Answer 28

P(B|A) = \frac{P(B)P(A|B)}{P(A)}

Question 29

What is the Law of Large Number?

Answer 29

For 𝑁 samples, the probability of one sample can be approximated by the empirical probability 𝑝 𝑋 ≈ 1/𝑁

Sample mean will approach actual mean as N approaches infinity

Question 30

What is the central limit theorem?

Answer 30

The Central Limit Theorem states, that a scaled version of the sample average is normally distributed about the true mean

Question 31

What are some additional distributions?

Answer 31

• X^2 distributions
• F distributions
• t- statistic / student-t distributions

Question 32

What is a type 2 error?

Answer 32

The sample shows correct when the statement is false

Question 33

What is type 1 Error?

Answer 33

The sample shows incorrect, when correct

Question 34

What is the hypothesis testing process?

Answer 34

1. Formulate hypothesis
2. Formally state hypothesis
3. Decide which relevant test-statistic, i.e. which metric 𝑇 shall be used in the decision-making process
4. Determine the distribution of the test-statistic, under the null hypothesis
5. Choose a level of significance 𝛼.
6. Under the null-hypothesis, with the distribution of the test-statistic, a threshold value for the data can be determined.
7. The threshold defines a “critical region” for the possible outcomes of the experiment, where 𝐻_0 is rejected
8. Compute the observed value of the test-statistic t
9. If 𝑡𝑜𝑏𝑠 is in the critical region, 𝐻0 can be rejected and 𝐻1 accepted.

Question 35

What is E[X]?

Answer 35

It is the expectation / first statistical moment

Question 36

How are the expectation and the variance of the variance of the x^2 distribution defined?

Answer 36

E[Z] = N
Var[z] = 2N

where N is the degrees of freedom

Question 37

How are the expectation and the variance of the variance of the t-statistic distribution defined?

Answer 37

E[T] = 0
Var[T] = 2/(N-2)

where N is the degrees of freedom