Linear models refresher

Question 1

What are deterministic models?

Answer 1

Given the same input, functions will return exactly the same output
e.g. y = mx +c

Question 2

What are statistical models?

Answer 2

they contain some pattern but there is variation around this pattern
e.g. given someone’s age we could attempt to predict their height

Question 3

What are linear models?

Answer 3

They are a type of statistical model

outcome = (model) + error

Question 4

Interaction terms

Answer 4

These occur when we can’t control for factors, so we include interaction terms to try and control for the effect on y
“as we move up 1 unit in x1, the slope of x2 changes by…”

Question 5

What is standard error?

Answer 5

how much we estimate our sample will vary due to random error

Question 6

what is the p-value

Answer 6

it is the likelihood of obtaining a result equal to or more extreme than one of our observations if the null is true
We use this to evaluate our null hypothesis
The smaller the p-value, the less likely the null is

Question 7

What is the sums of squares residual (SSresidual)?

Answer 7

the difference between the model line and each observed value of y
(y - y hat)
allows us to consider the model as a whole - rather than as specific coefficients

Question 8

What is the sums of squares total (SStotal)?

Answer 8

the difference between the mean of y and each observed value of y
(y - y bar)

Question 9

What is the sums of squares model (SSmodel)?

Answer 9

the difference between the model line and the mean of y
(y hat - y bar)

Question 10

What is R squared?

Answer 10

= how much variance in y is due to our predictors (we want this to be high)

R^2 = SSmodel / SStotal
R^2 = 1 - ( SSresidul / SStotal )

Question 11

what are joint tests?

Answer 11

they allow us to make inferences about the improvement of model fit with the inclusion of additional parameters (this allows us to test the reduction in SSresidual)

in R this is what the anova() function is for

Question 12

What are the assumptions of a linear model?

Answer 12

Our residuals (y - y hat) reflect everything we don’t account for in our models - in an ideal world these residuals would be just like random noise

We check how much ‘like randomness’ the residuals appear to be:
- mean of 0
- normal distribution
- constant variance ( = no detectable plotted pattern)
- independent and identically distributed

Question 13

What to do if our model does not meet assumptions:

Answer 13

• check if the model is mis-specified
• transform the outcome variable?
• bootstrap?

these will not help if the independence assumption is violated