L5 - Inferences from Predictions & Regressions

Question 1

When presented with 2 or more IVs to a prediction, what are the two possible ways to approach the analysis?

Answer 1

• Simple regression

- Multiple regression!

Question 2

Why would multiple regression be preferred over simple regression when there are multiple IVs?

Answer 2

By doing the regression as a multiple regression, the method of least squares partials out any joint overlapping effects on the DV among the set of IVs.

(Can be a problem when the IVs correlate with each other)/

Question 3

What were some things that went wrong when performing a simple regression for multiple IVs?

Answer 3

• R Sqaured values, when added together, were very high compared to the single R Squared value obtained from multiple regression
• Regression coefficient (R) values were not the same for each IV in multiple regression, when compared to separate simple regression coeffecients.

Question 4

How does multiple regression adjust the predictive effect of IVs to not have any overlap?

Answer 4

Method of least squares adjusts predictive effect of each IV in multiple regression due to CORRELATIONAL OVERLAP with over IVS by PARTIALLING IT OUT.

Each partial regression coefficient indicates the optimal strength of prediction for each IV, CONTROLLING for effects of all over IVs in the model.

Question 5

What are the two ways we can make an inference from linear regression from?

Answer 5

• Null Hypothesis testing

- Confidence intervals

Question 6

What is the corresponding population parameter to R Squared?

Answer 6

P ^2

(Rho) squared.

Question 7

What does the null hypothesis test for R Squared use?

Answer 7

ANOVA table

Question 8

How is mean square calculated?

Answer 8

MS = (the relevant) SS/ df

Question 9

What can be found in the ANOVA table?

Answer 9

• SS for SS reg, SS res, SS tot
• Df for reg, res, tot
• MS for reg, res, tot
• F test statistic (Tobs)
• p value

Question 10

What can be found in model summary statistics table?

Answer 10

• R (Multiple R)
• R square
• Adjusted R square

Question 11

On what distribution is the F statistic modelled on, from the ANOVA table?

Answer 11

F theoretical probability distribution

Question 12

What is the formula for F statistic (Tobs) in ANOVA

Answer 12

F = MS reg/ MS res

by looking at this, we can tell that F statistic is just a ratio for mean sums of regression and mean sums of residuals… hence it’s the AVERAGE AMOUNT OF VARIATION

Question 13

What is the shape of an F distribution defined by?

Answer 13

2 parameters: numerator DF (df reg), denominator DF (df res).

Question 14

What does the precision of a CI on R squared depend on?

Answer 14

• size of sample: bigger sample = more precise
• number of IVs: less IVs = more precise
• size of observed R square: larger R square = more precision

Question 15

How do we obtain a CI for Multiple R?

Answer 15

By square rooting the upper and lower onds of the CI for R square, as R = square root of R square.

Question 16

Interpret a 95% CI of [0.291, 0.789] for R Square

Answer 16

We are 95% confident that between 29% to 79% of the variance in the dependent variable at the population level is explained by the independent variables.

Question 17

What does it indicate if the observed R square value is actually outside the bounds of the Confidence Interval?

Answer 17

This indicates extreme bias in the observed R squared value, and may be due to a very small sample size and too many IVs.

Question 18

is observed R square biased?

Answer 18

Yes, it’s biased, but consistent.

So with greater sample size (and less Iv’s) it can be more accurate.

Question 19

What distribution does the unstandardised regression coefficient divided by its standard error follow (Tobs)?

Answer 19

t distribution.

like pearson correlations

Question 20

In a NHST of coefficients, what does the df for observed t statistic equal???

Answer 20

df = n - dfReg - 1.

where dfreg = number of IVS

so basically

df= n - IVs - 1

Question 21

How do we calculate the T obs for partial regression coefficient?

Answer 21

T obs = (observed regression coeff - null hyp value (zero))/standard error for regression coeff

Question 22

What can be found in a regression coefficients table?

Answer 22

Unstandardised coeffcient and it’s standard error.

standardised coefficients

t statistic, and it’s p value

and 95% CI on the unstandardised partial regression coefficient

Question 23

What are the assumptions of Linear Regression?

Answer 23

• residuals are NORMALLY DISTRIBUTED (no implication that scores on IV are normally distributed) - usually things can still work out unless extreme non-normality.
• variance of residuals is constant for any value of IV - HOMOSCEDASTICITY.
• scores on all variables are INDEPDANTLY and INDENTICALLY observed
• a LINEAR RELATIONSIP between DV and each IV
• variables are measured without error (hard to avoid in psychology).

Question 24

What are some consequences of failure to meet assumptions in linear regression?

Answer 24

affects:

• accuracy for F test for R square
• accuracy of standard errors for regression coefficients
• coverage of CI’s for R square and regression coefficients.
• unbiasedness of various sample statistics provided by analysis.

Question 25

How do we assess normality of residuals?

Answer 25

Assess Q-Q plot or histogram of the distribution of residual values for each person.

However, linear regression is robust to mild departures to normality.

Question 26

How do we asses homoscedasticity of residuals?

Answer 26

By creating a scatterplot, of standardised residuals vs standardised predicted scores on the dv. THEN ASSESS IF THERE IS ANY PATTERN AMONG DATA POINTS.

standardised to identify outliers more easily!

(standardised Predicted scores on x axis)

residual scores on y axis - use studentised deleted residuals. This means SD = 1

A residual outlier typically has a studentised deleted residual of 3 in medium to large samples, or 2.5 in smaller samples.

Question 27

What is homoscedasticity?

Answer 27

means that the variance of the residual values are the same either for any predicted score on the DV, or for any chosen score on the IV.

opposite is heteroscedasticity (on a continuum)

Question 28

What is heteroscedasticity

Answer 28

When there is systematic variability in the residual variances according to predicted values of the DV

Question 29

How is a residual outlier identified?

Answer 29

A residual outlier typically has a studentised deleted residual of 3 in medium to large samples, or 2.5 in smaller samples.

Question 30

What assumption is influential cases related to?

Answer 30

both non-normality and heteroscedasticity.

Question 31

How do we test for influential cases?

Answer 31

either:
1. Examine studentised deleted residuals for large absolute values
2. Cook’s d statistic –> If its value is 1 or more, shows excessive influence.

Question 32

What is Cooks d Statistic?

Answer 32

This measures the extent to which each case’s data values on the regression variables change the model parameters when that case is removed from the analysis, compared to when it is included.

minimum value = 0
large value (1+) = excessive influence

Question 33

How do we assess non-linearity of residuals?

Answer 33

Scatterplots: residuals vs predicted values.

We want to see a random scattering of data points, without any obvious pattern

(actually the same scatterplot as used for testing homoscedasticity).