Correlation & multiple regression

Question 1

what is correlation?

Answer 1

• An association or dependency between two independently observed variables
• use a scatterplot to visualise a correlation

Question 2

what does Pearsons correlation coefficient do?

Answer 2

• tells you how strong the correlation is between X and Y
• its a number between -1 and 1
• 0 they are completely independent of eachother
• 1.0 they are identical to eachther
• -1.0 they are exactly inverse of one another

Question 3

when is covariance greater?

Answer 3

when the values if X and Y and more similar

Question 4

when do we conduct a Pearsons’s coefficient (r)?

Answer 4

two interval/ration variables

Question 5

when do we conduct a Spearman’s rank coefficient

Answer 5

two ordinal (rank) variables

Question 6

when do we conduct a Kendall’s rank coefficient

Answer 6

two true dichotomy values

Question 7

when do we conduct a Phi coefficient?

Answer 7

two true dichotomy variables

Question 8

when do we conduct a point-biserial coefficient

Answer 8

one true dichotomy variable and one interval/ratio variable

Question 9

what is partial correlation?

Answer 9

when information from different variables is overlapping

Question 10

what is multiple regression?

Answer 10

it describes the relationship between one or more predictor variables (X1, X2 etc) and a single criterion (Y)

Question 11

linear regression equation

Answer 11

𝑌̂=𝛽0+𝛽1 𝑋1+𝛽2 𝑋2+…+𝛽𝑚 𝑋𝑚

𝒀̂ = the predicted value of the criterion variable 𝒀
𝜷𝟎 = the intercept term
𝜷𝒊 = the 𝑖th regression coefficient, indicating how strongly predictor variable 𝑿𝒊 can be used to predict 𝒀 in the model
𝒎 = the number of predictor variables in the model

Question 12

what is 𝑦=𝑎𝑥+𝑏 equivalent to?

Answer 12

𝑌̂=𝛽0+𝛽1 𝑋1
where a is the slope and b is the y intercept

Question 13

what is the equation for residual error?

Answer 13

𝜀=𝑌−𝑌̂

Question 14

what is the equation for the variance unexplained?

Answer 14

〖𝑆𝑆〗_𝑅=∑(𝑌−𝑌̂ )^2

Question 15

what is variance explained question?

Answer 15

〖𝑆𝑆〗_𝑀=∑(𝑌̂−𝑌̅ )^2

Question 16

what is prediction error?

Answer 16

the difference between the actual values 𝑌 and the predicted values 𝑌̂
𝜀=𝑌−𝑌̂

Question 17

what is the goal of a regression?

Answer 17

to find the best fit between the model and the observations, by adjusting the values of 𝛽_𝑖 until the prediction error is minimised

Question 18

what is multiple correlation coefficient (R)?

Answer 18

Correlation between the predicted values 𝒀̂ and the observed values 𝒀
- cannot directly be calculated
- has to be calculated by the square root of the coefficient of determination (R^2)

Question 19

what os the coefficient of determination (R^2)

Answer 19

• Proportion of variance of explained by the regression model
• This is simply the square of the multiple correlation coefficient

Question 20

F-ratio

Answer 20

the proportion of explained variance with the residual variance, allowing a statistical test

Question 21

effect size formultiple linear regression for cohen’s f

Answer 21

small effect size = cohen’s f of 0.02
medium effect size = cohen’s f of 0.15
large effect size = cohen’s f of 0.35

Question 22

what is a simultaneous (standard) multiple regression approach?

Answer 22

• No a priori model assumed
• All predictor variables are fit together

Question 23

what is a stepwise approach to multiple regression?

Answer 23

• No a priori model
• Predictor variables are added/removed one at time, to maximize fit
• Not a good approach because it will always overfit the data

Question 24

what is a hierarchical multiple regression approach?

Answer 24

• Based on a priori knowledge of variables – we may know a relationship exists for some variables, but are interested in the added explanatory power of a new variable
• Several subsequent regression models are analysed (adding or removing predictor variables)
• We can use this assess how much better one model explains the criterion variable than another (∆𝑅^2)

Question 25

what some factors that affect multiple linear regression?

Answer 25

• Outliers
• Scedasticity
• Singularity & Multicollinearity
• Number of observations /Number of predictors
• Range of values
• Distribution of values

Question 26

what are outliers?

Answer 26

• points which deviate substantially from most of the others can have a disproportionate effect on the linear regression fit

Question 27

what does cook’s distance measure?

Answer 27

the extremity of an outlier; values greater than 1 are cause for concern

Question 28

what is scedasticity?

Answer 28

• refers to the distribution of the residual error (i.e., relative to the predictor variable)
• Homoscedasticity: residuals stay relatively constant over the range of the predictor variable
• Heteroscedasticity: residuals vary systematically across the range of the predictor variable
• Multiple linear regression assumes homoscedasticity

Question 29

what is Multicollinearity?

Answer 29

refers to a high similarity between two or more variables (𝑟 > 0.9)

Question 30

what is Singularity

Answer 30

refers to a redundant variable; typically, this results when one variable is a combination of two or more other variables (e.g., subscores of an intelligence scale)

Question 31

issues with SINGULARITY & MULTICOLLINEARITY

Answer 31

• Logical: Don’t want to measure the same thing twice
• Statistical: Cannot solve regression problem because system is ill-conditioned

Question 32

how does the number of observations and number of predictions affect multiple regression?

Answer 32

• Number of observations (𝑁) should be high compared to the number of predictor variables (𝑚)
• Results become meaningless (impossible to generalise due to overfitting) as 𝑁/𝑚 decreases

Question 33

how does range and distribution affect multiple regression?

Answer 33

• Range:
  Small range (max-min) of the predictor variable restricts statistical power
• Distribution of variables:
  Data should be normally or uniformly distributed
  Always important to plot/visualise your data!