Regression

Question 1

What’s Gaussian quadrature and what’s disadvantages

Answer 1

Fitting n data points to a polynomial of order n+1

This will fit every data set without any errors, bad as data normally includes errors/outliers. Takes a lot of time/power, often desire able to fit to polynomial with fewer parameters than the data.

Question 2

What is the equation for the simple linear regression model

Answer 2

Y = ax + b + e

e error term (random variable, normally distributed with mean 0)
a slope
b intercept

Question 3

How do we find the coefficients in the simple linear regression equation

Answer 3

Rearrange for error term and find the ESS (error sum of the squares- sum all of the error values squared) and then partially differentiate the ESS wrt the different variables and set = 0 and solve the linear equations that result.

Question 4

What is the correlation coefficient, R

Answer 4

A measure of the strength of the linear relationship between two variables.

Close to 0 = uncorrelated
Positive = variables are positively correlated
Large = greater degree of correlation

For linear relationships it is equal to sq root of R^2

Question 5

How to modify ESS to get RSS, regression sum of the squares

Answer 5

Change yi to y bar (mean y value)

ESS represents error of the regressions estimate around its the actual value

RSS represents the error of the regressions estimate around the mean

Question 6

What is the TTS total sum of the squares

Answer 6

Sum (between i =1 & n): (yi - ymean)^2

Is equal to ESS + RSS

Represents the error of y around it’s mean

Question 7

What is the R^2 statistic

Answer 7

The fraction of the total variance (TSS) accounted for by the regression

R^2 = RSS/TSS

Closer to 1 shows the estimated regression function fits the data better

Useful even when non linear model is used where as R statistic is only useful for describing strength of linear relationships.