W2: Simple Linear Regression

Question 1

Regression

Answer 1

Deriving an equation for predicting one variable from another

Question 2

Regression model needs to:

Answer 2

How well can we predict Y, given a value of X?

How much variance in Y can we explain?

Question 3

Types of regression line

Answer 3

Simple linear regression: a single independent variable = the one IV

Multiple linear regression: multiple independent variables = all the IVs

Question 4

Total sums of squares (TSS)’s aim

Answer 4

Aim to explain or predict variability in y

Does not have anything to do with X

Question 5

How to know if the regression line is better predicting Y?

Answer 5

The closer the actual scores are to the predicted scores, the better the model predicts Y, the less variability around the line there is (better for generalisation)

The further away from the line (predicted scores) the actual scores are, the worse the model predicts Y, the more variability around the line there is

Question 6

What is residual?

Answer 6

Residual is the difference between predicted Y and actual Y for any given value of X

Question 7

Residual SS vs Regression SS

Answer 7

Residual sums of square → small value

Regression sums of squares → big values

Question 8

Where abouts are the negative and positive residual?

Answer 8

(+) residual: above the regression line
(-) residual: below the regression line

Question 9

Why does the residual sum always equal to 0? And what would happen to the sum if there is an outlier?

Answer 9

Residual sum would always be 0 - because there are half that are negative and half that are positive (regression line sits in the middle of data points)

Even with an outlier, the regression sum would be 0 as the regression line would skew to compensate and adjust

Question 10

Conditional and marginal distribution

Answer 10

Marginal distribution of Y: spread or variance of scores around the mean
* Wider and more spread out → more variability compared to the conditional distribution

Conditional distribution of Y: spread of variance of scores around the regression line, for any given value of X
* Narrower and tighter

Question 11

R^2: coefficient of determination

Answer 11

Is split into regression and error

If variability is explained → regression

If viability not explained → residual

Question 12

R^2 effect:

Answer 12

R^2 ranges from 0-1 (often expressed as a percentage)
Closer to 0 (or 0%: the weaker the effect, the less variance the model explains
Closer to 1 (or 100%): the stronger the effect, the more variance that the model explains

Question 13

Different ways to find correlation

Answer 13

pwcorr or The square root of R^2→ the correlation between the predicted values of Y and the actual values of Y

Question 14

What are the effects tested in regression? What statistical signifance is used to test it?

Answer 14

Effects that are teste:
* Model-as-a-whole
* Individual variable or predictor effects

Statistical significance:
* Model as a whole: F ratio and p-value
* Individual prediction: t-statistics and p-value

Question 15

How is mean square (MS) calculated?

Answer 15

sums of squares divided by degrees of freedom

Question 16

F Ratio (Model as a whole) H0 and H1

Answer 16

H0 (null hypothesis): the model is no better than the intercept only model (aka the null model)
H0: all regression coefficients = 0

H1 (alternative hypothesis): the regression model significantly better than the null model
H1: at least on regression coefficient is not equal to 0

Question 17

T statistics ( Individual variable) H0 and H1

Answer 17

H0: b=0
H1: b≠0

Question 18

Statistical signifiance of p-value

Answer 18

p<.05

Question 19

Model as a whole R^2 effect size

Answer 19

R-squared
* 2% - 12%: small effect
* 13% - 25%: medium effect
* 26% and above: large effect

Question 20

Individual variable statistacal significant via beta coefficient

Answer 20

Beta coefficient: unstandardised, no standard cut offs
Need to understand the variable’s scales to know whether it is big or small effect
Can tell whether of not significance by p-value
Beta coefficient: standardised
Interpret similarly to correlation coefficient

Question 21

Why is there a need for standardised regression coefficient?

Answer 21

Because unstandardised effect size are useful if the ‘natural’ scale is known
* Without a natural scale, direct comparison cannot be made as variables may be on different scales
* One-point increase can be a small small change or a big change (e.g. one point on a 1-7 scale is bigger than one point on a 0-100 scale)