Statistic

Question 1

What do we mean by the population of study?

Answer 1

The population is the set of sources from which data has to be collected.

Question 2

What is a sample?

Answer 2

A Sample is a subset of the Population being studied.

Question 3

What is a variable?

Answer 3

A Variable is any characteristics, number, or quantity that can be measured or counted. A variable may also be called a data item.

Question 4

What is a statistical parameter?

Answer 4

Also known as a statistical model, A statistical Parameter or population parameter is a quantity that indexes a family of probability distributions. For example, the mean, median, etc of a population.

Question 5

What are the two data type ?

Answer 5

Numerical: data expressed with digits; is measurable. It can either be discrete (finite number of values) or continuous (infinite number of values).

Categorical: qualitative data classified into categories. It can be nominal (no order) or ordinal (ordered data).

Question 6

What is the mean, median and mode?

Answer 6

Mean: the average of a dataset.

Median: the middle of an ordered dataset; less susceptible to outliers.

Mode: the most common value in a dataset; only relevant for discrete data.

Question 7

What is the range?

Answer 7

Range: the difference between the highest and lowest value in a dataset.

Question 8

What is the variance, its properties and its formula?

Answer 8

Variance (σ2): measures how spread out a set of data is relative to the mean.

Var(X) = E[(X-E(X))²]

Var(aX+b) = a²Var[X]

Var[X+Y] = Var[X]+Var[Y]+2Cov[X,Y]

Question 9

What is R squared?

Answer 9

R-Squared: a statistical measure of fit that indicates how much variation of a dependent variable is explained by the independent variable(s); only useful for simple linear regression.

Question 10

What is the covariance, its properties and its formula?

Answer 10

Covariance: Measures the variance between two (or more) variables. If it’s positive then they tend to move in the same direction, if it’s negative then they tend to move in opposite directions, and if they’re zero, they have no relation to each other.

Cov[X,Y] = E[XY]-E[X]E[Y] (is zero if X and Y are independant)

Cov[X,Y]=E[(X-E(X))(Y-E(Y))]

Question 11

What is the correlation and its formula?

Answer 11

Correlation: Measures the strength of a relationship between two variables and ranges from -1 to 1; the normalized version of covariance. Generally, a correlation of +/- 0.7 represents a strong relationship between two variables. On the flip side, correlations between -0.3 and 0.3 indicate that there is little to no relationship between variables.

Question 12

What is a probability density function (pdf)?

Answer 12

Probability Density Function (PDF): a function for continuous data where the value at any point can be interpreted as providing a relative likelihood that the value of the random variable would equal that sample.

Question 13

What is a probability mass function (pmf)?

Answer 13

Probability Mass Function (PMF): a function for discrete data which gives the probability of a given value occurring.

Question 14

What is a cumulative density function (CDF)?

Answer 14

Cumulative Density Function (CDF): a function that tells us the probability that a random variable is less than a certain value; the integral of the PDF.

Question 15

What is the moment of a distribution?

Answer 15

Moments describe different aspects of the nature and shape of a distribution. The first moment is the mean, the second moment is the variance, the third moment is the skewness: and the fourth moment is the kurtosis.

Question 16

What is the skewness of a distribution ?

Answer 16

Skewness is the third central moment of a distribution. It is the measure of the lopsidedness of the distribution; any symmetric distribution will have a third central moment, if defined, of zero. The normalised third central moment is called the skewness, often γ. A distribution that is skewed to the left (the tail of the distribution is longer on the left) will have a negative skewness. A distribution that is skewed to the right (the tail of the distribution is longer on the right), will have a positive skewness.

Question 17

What is the kurtosis of a distribution?

Answer 17

Its the fourth central moment. It is a measure of the heaviness of the tail of the distribution, compared to the normal distribution of the same variance. Since it is the expectation of a fourth power, the fourth central moment, where defined, is always nonnegative; and except for a point distribution, it is always strictly positive.

Question 18

What do we mean by probability?

Answer 18

Probability is the likelihood of an event occurring.

Question 19

What do we mean by a independant event ?

Answer 19

Independent events are events whose outcome does not influence the probability of the outcome of another event; P(A|B) = P(A).

Question 20

What do we mean by mutually exclusive event?

Answer 20

Mutually Exclusive events are events that cannot occur simultaneously; P(A|B) = 0.

Question 21

What do we mean by conditional probability? What its formula?

Answer 21

Conditional Probability [P(A|B)] is the likelihood of an event occurring, based on the occurrence of a previous event.

Question 22

What is bayes theorem and its formula?

Answer 22

Bayes’ Theorem: a mathematical formula for determining conditional probability. “The probability of A given B is equal to the probability of B given A times the probability of A over the probability of B”.

Question 23

When talking about hypothesis testing, what do we means by null hypothesis, alternative hypothesis, p-value, alpha and beta.

Answer 23

Null Hypothesis: the hypothesis that sample observations result purely from chance.

Alternative Hypothesis: the hypothesis that sample observations are influenced by some non-random cause.

P-value: the probability of obtaining the observed results of a test, assuming that the null hypothesis is correct; a smaller p-value means that there is stronger evidence in favor of the alternative hypothesis.

Alpha: the significance level; the probability of rejecting the null hypothesis when it is true — also known as Type 1 error.

Beta: type 2 error; failing to reject the null hypothesis that is false.

Question 24

What are the 4 step of hypothesis testing?

Answer 24

Steps to Hypothesis testing:

1. State the null and alternative hypothesis
2. Determine the test size; is it a one or two-tailed test?
3. Compute the test statistic and the probability value
4. Analyze the results and either reject or do not reject the null hypothesis (if the p-value is greater than the alpha, do not reject the null!)

Question 25

What is the expected value and its basic properties?

Answer 25

The expected value of a discrete random variable is the probability-weighted average of all its possible values.

Its basic properties are:

E[X]= Σx_ip_i

E[X+Y] = E[X]+E[Y]

E[aX+b] = aE[X] + b

E[XY] = E[X]E[Y] iff X and Y are independant

Question 26

How are the variance and mean related ?

Answer 26

Var(X) = E[X²]-(E[X])² = E[(X-u)²]

Question 27

What are the pdf, mean, variance and domain of X of these discrete distributions: Geometric, Uniform, Binomial, Bernoulli,Hypergeometric, Negative Binomial and Poisson?

Answer 27

Question 28

Explain the logic behind these discrete distributions: Geometric, Uniform, Binomial, Bernoulli,Hypergeometric, Negative Binomial and Poisson?

Answer 28

Geometric: The probability distribution of the number X of Bernoulli trials needed to get one success.

Uniform: every one of n values has equal probability 1/n.

Binomial: the binomial distribution with parameters n and p is the discrete probability distribution of the number of successes in a sequence of n independent experiments.

Bernoulli: the set of possible outcomes of any single experiment that asks a yes–no question.

Hypergeometric: Describes the probability of x successes in n draws, without replacement, from a finite population of size N that contains exactly r objects with that feature. Where in each draw is either a success or a failure.

Negative Binomial: is the number of successes in a sequence of independent and identically distributed Bernoulli trials before a specified (non-random) number of failures (denoted r) occurs.

Poisson: expresses the probability of a given number of events occurring in a fixed interval of time or space if these events occur with a known constant mean rate and independently of the time since the last event

Question 29

What is the exponential distribution? What are the mean, variance and pdf? What does it pdf look like?

Answer 29

the exponential distribution is the probability distribution of the time between events in a Poisson point process, i.e., a process in which events occur continuously and independently at a constant average rate.

pdf: f(x,λ) = λe^-λx x>= 0

mean = 1/λ

Variance: 1/λ²

Question 30

What is the normal distribution? What its pdf, mean and variance? What does it look like?

Answer 30

A normal distribution is a symmetrical distribution sometimes informally called a bell curve (Gaussian).

Mean: u

Variance: σ²

pdf:

Question 31

What do we mean by information entropy?

Answer 31

The information entropy, often just entropy, is a basic quantity in information theory associated to any random variable, which can be interpreted as the average level of “information”, “surprise”, or “uncertainty” inherent in the variable’s possible outcomes.

Question 32

What does information theory study ?

Answer 32

Information theory studies the quantification, storage, and communication of information.

Overview: Information theory studies the transmission, processing, extraction, and utilization of information. Abstractly, information can be thought of as the resolution of uncertainty. In the case of communication of information over a noisy channel, this abstract concept was made concrete in 1948 by Claude Shannon in his paper “A Mathematical Theory of Communication”, in which “information” is thought of as a set of possible messages, where the goal is to send these messages over a noisy channel, and then to have the receiver reconstruct the message with low probability of error, in spite of the channel noise.

Question 33

What is the main idea of the central limit theorem (CLT) and how can it be applied to the sample mean?

Answer 33

In probability theory, the central limit theorem (CLT) establishes that, in some situations, when independent random variables are added, their properly normalized sum tends toward a normal distribution (informally a “bell curve”) even if the original variables themselves are not normally distributed.

For example let take the sample mean: S_n = (X₁+X₂+…+X_n)/n

The sample mean is a random variable. The usefulness of the theorem is that the distribution of √n(Sn − µ) approaches N(0,σ²) regardless of the shape of the distribution of the individual Xi.

Question 34

What is a combination and what is its formula?

Answer 34

A combination is similar to a permutations. However, the order of the selected items does not matter. For example, the arrangements ab and ba are equal in combinations.

Question 35

What is a permutation? What are the formula for the total number of permutation with and without repetition?

Answer 35

A permutation is an ordered combination.

Let say we have N item and pick r of them. When repetition is allowed the total number of permutation is: P(n,r) = n^r

When repetition is not allowed the total number of permutation is:

P(n,r) = n!/(n-r)!

Question 36

What is an estimator?

Answer 36

In statistics, an estimator is a rule for calculating an estimate of a given quantity based on observed data. The estimator itself is a random variable.

Question 37

What is the difference between a point estimator and a interval estimator?

Answer 37

The point estimators yield single-valued results, although this includes the possibility of single vector-valued results and results that can be expressed as a single function.

This is in contrast to an interval estimator, where the result would be a range of plausible values (or vectors or functions).

Question 38

What are the four quantified properties of an estimator? What is the link between MSE, Bias and Variance?

Answer 38

Note, i use o to represent the estimator generally it is theta.

1) Error: For a given sample x, the error of the estimator ô is defined as

e(x) = ô(x)-o

2)Mean squared error, is the probability-weighted average of the sqaured erros:

MSE(ô) = E[(ô(X)-o)²]

3) Variance: It is used to indicate how far, on average, the collection of estimates are from the expected value of the estimates. Keep in mind the estimator is a random variable.

Var(ô)=E[(ô-E(ô))²]

4) Bias is the distance between the average of the collection estimates, and the single parameter being estimate. It is defined as:

B(ô) = E(ô)-o

5) Relationships among the quantites:

MSE(ô)= var(ô) + (B(ô))²

Question 39

What do we mean by a consistent estimator ?

Answer 39

A consistent sequence of estimators is a sequence of estimators that converge in probability to the quantity being estimated as the index (usually the sample size) grows without bound. In other words, increasing the sample size increases the probability of the estimator being close to the population parameter. Mathematically we have:

Question 40

What do we mean by a Asymptotically normal estimator ?

Answer 40

An asymptotically normal estimator is a consistent estimator whose distribution around the true parameter θ approaches a normal distribution with standard deviation shrinking in proportion to {\displaystyle 1/{\sqrt {n}}} as the sample size n grows. Mathematically we have:

Question 41

What do we mean by an efficient estimator?

Answer 41

An efficient estimator, is an estimator having the lowest variance. In other word, it extract the optimal amount of information from the data.

Question 42

What do we mean by a Robust estimator/statistic?

Answer 42

Robust statistics are statistics with good performance for data drawn from a wide range of probability distributions, especially for distributions that are not normal.

Question 43

What do we mean by parametric statistics?

Answer 43

Parametric statistics is a branch of statistics which assumes that sample data come from a population that can be adequately modeled by a probability distribution that has a fixed set of parameters.

Question 44

What do we mean by non-parametric statistics?

Answer 44

Nonparametric statistics is the branch of statistics that is not based solely on parametrized families of probability distributions (common examples of parameters are the mean and variance). Nonparametric statistics is based on either being distribution-free or having a specified distribution but with the distribution’s parameters unspecified. Nonparametric statistics includes both descriptive statistics and statistical inference.

Question 45

What is the difference between Parametric probability density estimation and nonparametric probability density estimation?

Answer 45

Parametric probability density estimation involves selecting a common distribution and estimating the parameters for the density function from a data sample.

Nonparametric probability density estimation involves using a technique to fit a model to the arbitrary distribution of the data, like kernel density estimation.

Question 46

What are the two step to estimate the pdf of a parametric distribution and how can we check if it is a good fit?

Answer 46

The shape of a histogram of most random samples will match a well-known probability distribution. The common distributions are common because they occur again and again in different and sometimes unexpected domains. Once identified, estimate the parameter of the distribution. For example, if it look like a normal we need the mean and variance.

To verify if it a good fit we can:

1) Plot the density function and comparing the shape to the histogram.
2) Sample the density function and comparing the generated sample to the real sample.
3) Use a statistical test to confirm the data fits the distribution.

Question 47

When should we perform a non-parametric density estimation?

Answer 47

In some cases, a data sample may not resemble a common probability distribution or cannot be easily made to fit the distribution.

This is often the case when the data has two peaks (bimodal distribution) or many peaks (multimodal distribution).

In this case, parametric density estimation is not feasible and alternative methods can be used that do not use a common distribution. Instead, an algorithm is used to approximate the probability distribution of the data without a pre-defined distribution, referred to as a nonparametric method.

Question 48

What is a kernel density estimation?

Answer 48

In statistics, kernel density estimation (KDE) is a non-parametric way to estimate the probability density function of a random variable. Kernel density estimation is a fundamental data smoothing problem where inferences about the population are made, based on a finite data sample.

Question 49

What is bootstrapping?

Answer 49

In statistics, bootstrapping is any test or metric that relies on random sampling with replacement. Bootstrapping allows assigning measures of accuracy to sample estimates.