Statistics

Question 1

Covariance

Answer 1

Covariance is a measure of the joint variability of two random variables. It is positive when the values of the two variables increase or decrease together, negative when the values increase or decrease oppositely, or zero when the two variables are completely linearly uncorrelated.

Formula:

Question 2

Pearson correlation coefficient

Answer 2

Pearson correlation coefficient measures the linear correlation between two random variables. It is the covariance normalized by the standard deviations of the two variables. It is 1 for a total positive linear correlation, 0 for no linear correlation, and -1 for a total negative linear correlation.

Formula:

Question 3

Covariance matrix

Answer 3

Covariance matrix is a square matrix giving the covariances between each pair of elements of a random vector. The main diagonal contains the covariances of an element with itself (i.e. the variances).

Illustration:

Question 4

Laplace Smoothing

Answer 4

Laplace smoothing (aka additive smoothing) is a technique used to smooth categorical data.

Example application in calculating the probabilities of word frequencies:

Question 5

Parametric Model

Answer 5

A parametric model is a model with a fixed number of parameters regardless of the input size (i.e. number of training examples). As a consequence, the form of the learned function is predefined.

Linear regression and neural networks are examples of parametric models.

Question 6

Non-parametric Model

Answer 6

A non-parametric model is a model whose number of parameters adapt to the input size (i.e. number of training examples). As a consequence, the form of the learned function is flexible.

K-Nearest Neighbors and Support Vector Machines are examples of non-parametric models.