L2 - The Normal and Associated Distributions

Question 1

If X is a continuous random variable, What is P(X=1)?

Answer 1

- An accurate (but unhelpful) answer to this question
is P(X=1)=0. In fact 𝑃(𝑋=𝑎)=0 at any given a within the interval.
- Why? - as it is just one point we cannot calculate the area - it is an infinitesimally small point with such as infinitesimally small area which we say is 0

∫_-∞^∞f(x)=1 (total probability)

Question 2

What is a continuous random variable?

Answer 2

• For example, we might wish to measure the temperature in a particular location over a period of time.
• Alternatively, we might wish to measure the distance between the place of residence and the place of work for an individual.
• In both these cases the random variable is more
  naturally thought of as lying somewhere on a continuum of possible values rather than taking one of a discrete number of possibilities.
• When the number of outcomes for a discrete distribution is large
  then we can often approximate it by a continuous distribution.

Question 3

What criteria must you satisfy to become a probability density function?

Answer 3

PDF example –> p(a ≤ X ≤ b) = ∫_a^b f(x) dx

• f(x) ≥ 0
• ∫_-∞^∞f(x)dx = 1

Question 4

What is the z transformation or standardising a normal distribution?

Answer 4

• It is possible to transform any normal distribution into the standard normal distribution (mean = 0, standard deviation = 1) as follows:
• X~N(μ,σ^2)
• Z=(X-μ)/σ ~ N(0,1)
• This is useful because we can take data from different sources onto the same scale and only have to tabulate the standard normal distribution to be able to look up critical values and/or p-values for test statistics.

Question 5

What is the function normal distribution?

Answer 5

• f(x) = (1/σsqrt(2π)) * exp[-((x-μ)^2)/(2σ^2))]

Question 6

What is a useful deature of the normal distribution?

Answer 6

• is that linear combinations of normally distributed random variables will themselves follow a normal distribution
• For example, let X{2} ~N(μ{1},σ{1}^2) and X{2}~N(μ{2},σ{2}^2) be independent normal random variables
• If a and b are constants then a linear combination of the variables using a and b as weights has the following normal distribution:
• aX{1} + bX{2} ~ N(aμ{1}+bμ{2}, a^2σ{1}^2 + b^2σ{2}^2)
• The normal distribution is unique in having this property and therefore, if we can assume normality, this is very useful in deriving the distribution of random variables which are functions of other random variables

Question 7

What are moments of distribution?

Answer 7

• are the expectations of integer powers of the random
  variable in question
• For example, if X is a random variable, then its first three moments are E(X), E(X^2) and E(X^3) .
• These are the raw moments of the distribution

Question 8

What are central moments?

Answer 8

• the central moments which are the expectations of the deviation of the random variable from its mean (or first moment).
• Thus the second central moment of the random variable X can be written as E(X -E(X))^2= σ^2 which is the variance of x.

Question 9

What are some examples of higher order moments?

Answer 9

Higher order moments are often scaled by the standard deviation to obtain measures such as:

• skewness –> E(X - E(X))^3/σ^3
• kurotosis –> E(X - E(X))^4/σ^4
• These measures are useful in characterising the shape of a distribution and are often referred to as the moments of the distribution even though, strictly speaking, they are transformations of the raw moments.

Question 10

What is the mean function for continuous distribution?

Answer 10

μ= E(X)= ∫_-∞^∞f(x)x dx

Question 11

What is the variance function for continuous distribution?

Answer 11

• σ^2= E(X-E(X))^2= E(X^2- E(X)^2) = ∫_-∞^∞(x-μ)^2 *f(x) dx

Question 12

How do you find out the function of higher order moments for continous distribution?

Answer 12

• can be calulcuated by integrating a function of the form

- ∫_a^bf(x) (x-E(x))^k *f(x) dx (where a and b are the minimum and maximum possible values) and scaling by σ^k

Question 13

What is the Chi-squared distribution?

Answer 13

• Is used to test the ‘goodness of fit’ a theoretical model is to a observed one
• so looking at the variance of the residual error (residual sum of squares) or a regression model for the actual data
• Also looking at the probability you could get those errors while holding some variables constant.
• The Chi-squared is derived or sampled from normal distribution with the formula - if a variable is Z is distributed normally it is said that Z^2 will have a Chi-squared distribution :
• χ = Σ_j=1^k (Z_j^2)
• also calculated:
  χ = Σ(O-E)^2/E
• Z is the risdual of a model
• the random variable defined by this is said to follow a chi-squared distribution with k degrees of freedom.
• When looking up on a table the P-value is the probability of it be larger than that value
• used when one variable depends on another when hypothesis test??? e.g. amount of women and men in a sample depends on their ages?

Question 14

What does the Chi-squared distribution look like on a graph?

Answer 14

• if degrees of freedom (k) = 0, 1 or 2 –> chi squared no longer has a PDF which takes the value 0 at x=0 –> instead the value of the PDG tends to infinity as x(chi) tends to zero –> look like 1/x graph for k=1 and is downwards sloping for k=2
• if k > 2 –> The PDF takes the value 0 for x=0, reaches a single peak for some value of x >0 and declines asymptotically to 0 as x becomes large. - it is positively skewed.
• you get a negative gradient line at the start because as you are squaring all values of a standard normal distribution you are getting rid of all negative values, and at lower degrees of freedom this creates the negative gradient curve

Question 15

What is the mean and variance for Chi-squared distribution?

Answer 15

mean –> k

variance –> 2k

Question 16

What happens when k is large for a Chi-squared distribution?

Answer 16

that as k becomes large, the asymmetry which we have
observed in the PDF of the chi-squared distribution becomes less pronounced. In the limit, for large qk, the chi-squared distribution will look more and more like the normal distribution (as is predicted by the central limit theorem).

Question 17

What is the F-distribution?

Answer 17

• Suppose we have two random variables each of which follows a chi-squared distribution
• A variable follows an F-distribution if it is constructed as the ratio of two Chi-squared distributed variables each of which is divided by its degrees of freedom:

let X{1] ~ χ{k{1}}^2 and X{2}~χ{k{2}}^2

X = (X{1}/k{1})/(X{2}/k{2}) = (X{1}/X{2}) * (k{2}/k{1}) ~ F{k{1},k{2}}

• The F-distribution arises naturally in econometric analysis when we consider the ratios of variables which are constructed as the sum of squared random variables
• comparing the s.d. or variance of two sets of data to see if they are statistically different to each other - more variation in errors from the theortical values
• the value given in the table give you F values which if are greater than the critical value (usually 5%) it is rejected
• Note that the order of these degrees of freedom is important:
  F{k{1},k{2}}^5% ≠ F{k{2},k{1}}^5%

Question 18

What is the shape of the F distribution?

Answer 18

• For k{1}>4 the F distribution has a similar shape to the chi-squared distribution.
• The F distributions with k{1} less than 4 does not have the typical shape:
• For k{1}= 3 –> f(0) >0 while for χ= 1 or 2, f(χ) –> ∞ as χ –> 0 - rather like we saw for the chi-squared distribution –> again similar shape.
• Another similarity with the chi-squared distribution is that as both k{1} and k{2} become large, the shape of the F distribution becomes symmetric.

Question 19

What is the Student’s t-distribution?

Answer 19

Suppose X{1} is a random variable that follows a standard normal distribution i.e. X~N(0,1) and X{2} is a
random variable that follows a chi-squared distribution with k degrees of freedom i.e. X{2}~χ{k}^2. It can be shown that the random variable defined:

X ~ (X{1}/sqrt(X{2}/k))

• Student’s t distribution is often referred to simply as the t distribution. It arises in
  econometrics (and in many other statistical situations) when we wish to conduct hypothesis tests on a variable which we assume is normally distributed but for which we do not know `the variance

Question 20

How else can you write the student t-distribution?

Answer 20

• if n observations were take nfrom normal distribution with mean μ and variance σ^2, the sample mean X(bar) a N(μ,σ^2/n) distribution that Z = (X(bar) - μ)/(σ/sqrt(n)) followed a N(0,1^2) distribution.
• If σ was unknown then S, where S^2 was an unbiased estimator of σ, was used.
• Then providing n was large,
  (X(bar) - μ)/(S/sqrt(n)) ≈ N(0,1^2)
• However, if n is small, S is unlikely to be very close to σ and (X(bar) - μ)/(S/sqrt(n)) can no longer be modelled by the normal distribution N(0,1^2)
• When n is small we usually use the symbol t to denote the quantity (X(bar) - μ)/(S/sqrt(n))
• If a random sample X{1}, X{2} …, X{n} is selected from a normal distribution with mean μ and unknown σ ^2 then:
• t= (X(bar) - μ)/(S/sqrt(n)) has a t{n-1} distribution where:
• S^2 = (ΣX^2 - nX(bar)^2)/n -1

Question 21

What does the student t-distribution look like?

Answer 21

• The shape of the PDF of the t distribution looks very much like that of the standard normal distribution. It is symmetric around zero and has the characteristic ‘bell shape’ of the normal distribution.
• However, the t-distribution has ‘fatter tails’ than those of the normal distribution.
• By this we mean that more of the mass of the distribution lies in its tails than is the case of the normal. This means that ‘extreme events’ (or values of the random variable that lie in the tails) are more likely for the t distribution
• as the the degrees of freedom gets large the t-distribution tends towards a normal distribution

Question 22

When is the student distribution useful?

Answer 22

• The t-distribution is most useful when constructing tests based on small samples. - As the sample size gets larger the differences between the t distribution and the normal get smaller.
• In the limit, as the sample size becomes arbitrarily large, the t distribution converges on the
  normal.
• In practice, for sample sizes more than 30, it is very difficult to tell these two
  distributions apart.