Lecture 3 - Correlations and Regression

Question 1

Define variability and covariability.

Answer 1

Variability is much a given varies from observation to observation.

Covariability refers to how much two variables vary together.

Question 2

Correlation does not necessarily imply causation.

True or false?

Answer 2

True.

Question 3

How to determine the degrees of freedom when doing a correlation?

Answer 3

df = n - 2

Question 4

Why is sample size so important when doing correlation analysis?

Answer 4

Because a small sample size could easily yield a correlation that is not actually reflective of the population.

Question 5

What is the difference between correlation and regression?

Answer 5

Correlation refers to whether there is an association between variables and regression is the predictive model of that association.

Question 6

In regards to regression what is the least squares solutions?

Answer 6

The least squares solution is how the line of best fit (regression model) is determined.

Question 7

What is homosedasticity?

Answer 7

Equal variance across observations.

Question 8

What is the difference between correlation and regression?

Answer 8

Correlation refers to the association between two variables.
Regression refers to predicting one variable from another (if in fact we can predict one variable from another).

Question 9

What is Regression?

Answer 9

With regression, we are not asking is there a relationship between two variables, we are asking what is the best linear relationship to describe that association. This is often expressed as y = a + bx

Question 10

In regards to the typical equation relating to Regression in psychology, where y = a + bx + e, what do the different coefficients refer to?

Answer 10

y - the outcome variable
x - predictor variable
a - intercept parameter (sometimes called the constant)
b - slope parameter
e - error or residual term

Question 11

With regards to regression in psychology, what are the conditions/assumptions made about the error included in a regression term? i.e. the “e” in y = a + bx?

Answer 11

The error or residual term is assumed to be;
1. independent
2. Normally distributed - with a mean of zero
3. Homescedastic - equal error variance for predicted values of Y

Question 12

With regard to Regression, what is the Least Squares Solution?

Answer 12

The Least Squares solution is how the “line of best fit” is determined from observed data, where the the regression line is determined by summing together the error between a predicted value and the observed value, such the when the the differences are squared and summed the the regression line has the smallest value for the sum of squares - hence the “least squares solution” being what this is referred to as.

Question 13

How is a regression line determined from a set of data?

Answer 13

The Least Squares Solution is how a regression is determined/arrived at.

Question 14

Does the slope in a regression model reflect the correlation between the two variables?

Answer 14

No. The slope does not predict or reflect correlation.
You could have a steep slope and a poor correlation, or a shallow slope and a strong correlation. The slope is simply a results of the least squares solution used to determine the line of best fit.
The direction of the slope, however, does reflect the correlation, such that if there is a downwards slope then this is reflective of a negative correlation, whereas if there is an upward slope this is reflective of a positive correlation

Question 15

In Regression, what does “error” refer to?

Answer 15

Error refers to the difference in what the predicted outcome and the observed outcome.

Question 16

Should a regression always report the variance explained? And if so, what does this mean?

Answer 16

Yes.
Variance explained simply refers to the idea that the regression proposed for a set of data between two variables should tell us how much of the variance between two variables is explained by the correlation/relationship between these variables or by the predictor. Variance explained is understood at as the R-squared value, which can be obtained from JASP output.

Question 17

What is the ANOVA (F test)?

Answer 17

The F test tells us whether the variance explained is significantly different from zero and therefore whether the variance explained (r-squared) given with regression indicates a significant relationship between two variables. Another way of saying this is that the F test tells us whether a variable is a significant predictor for the outcome variable.

Question 18

What does the F test for regression tell us?

Answer 18

The F test answers the question: does the regression line help to explain the variance in Y? Where the Null hypothesis would say that r-squared is = 0

Question 19

Does r-squared and F (ANOVA) testing tell us the strength of a relationship/correlation?

Answer 19

No. The r-squared value and whether it is significant or not does NOT tell us the strength or direction of a relationship.

Question 20

What are the regression coefficients?

Answer 20

If we think about regression as y (outcome variable) = a + bx(predictor variable), then the regression coefficients are a and b (as they were taught to us in the lecture), where b is the size and the direction of the slope and a is the y intercept.

Question 21

When measuring the correlation between two variables using regression, is it always necessary to determine whether the y-intercept is significant?

Answer 21

No. The y - intercept refers to the value of y when x = 0. Depending on the variables, this may not always be a meaningful measure. An example of this response times or intelligence. In these examples the predictor variable can never be 0.

Question 22

What does Pearson’s r measure?

Answer 22

Pearson’s correlation coefficient (r) measure the linear association between two continuous variables. The Pearson’s correlation coefficient measures how much two variables vary together (covariability) and much the vary separately(variability).
The equation for Pearson’s correlation coefficient r is: r = covariability of X and Y/Variability of X and Y separately

Question 23

When looking at whether there is a linear correlation between two continuous variables, we can use Pearson’s Correlation Coefficent (r). How does sample size affect whether we will see a significant value of r?

Answer 23

With Pearson’s r, we need a large sample size (or vary strong correlation) in order to see a significant r value, as r is dependent on degrees of freedom (where degrees of freedom in this case is n - 2).

Question 24

What are the degrees of freedom for Pearson’s Correlation Coefficient?

Answer 24

Degrees of freedom for Pearson’s correlation coefficient are n - 2.

Question 25

What is Spearman’s Correlation?

Answer 25

Spearman’s correlation is used to measure whether there is a correlation between variables where the data is one-directional, but not linear. In order to use Spearman’s correlation the data needs to be made ordinal/ranked - doing this makes non-linear data linear.

Question 26

What is the benefit of using Spearman’s correlation coefficient?

Answer 26

Sometimes we have data that suggest a correlation between variables, but the relationship does not appear to be linear, i.e. it may be a non-linear relationship. Using Pearson’s r we may find that there is no relationship between the variables, but there may be a non-linear relationship. Spearman’s correlation allows to determine whether there may be a relationship between our variables that is non-linear. In order to determine whether this is the case the data need to be made ordinal/ranked. The process from there is then similar to Pearson’s r, as when we rank the data, if there is a relationship, it will will be linear.

Question 27

What is Cronbach’s alpha?

Answer 27

Cronbach’s alpha is a measure of consistency of measure. Or the reliability of a measure. Cronbach’s alpha is measure of internal consistency and a measure of scale reliability. Cronbach’s alpha is calculated by taking the average covariance and dividing it by the average total variance. Therefore, in order to have a high alpha there needs to be a high level of covariance. In other words, we need the covariance to explain the majority of the variance seen.

Question 28

Are correlation and regression the same?

Answer 28

No. Correlation tells us whether there is an association between two continuous variables.
Regression refers to the linear relationship between two variables that best describes the association such that predictions about one variable can be made based on the value of the other variable. This can be expressed as y (outcome variable) = a (intercept) + b(slope) x (predictor variable), i.e. y = a + bx

Question 29

What does the regression coefficient b (slope) tell us about the association/correlation between two variables?

Answer 29

b or the slope tells us:
1. whether there is a relationship between X and Y
2. Whether the relationship is positive or negative
3. Estimate/prediction of expected change in Y when X increases by 1

Question 30

What kind of statistical test is used to determine whether the regression coefficient r is significant or not?

Answer 30

A t-test is used and JASP/SPSS can determine whether it is significant or not.

Question 31

When doing a regression analysis there are assumptions that need to be met. What are the assumptions about errors that are made when doing a regression analysis?

Answer 31

1. The errors are independent (this has more to do with how data are measured and collected)
2. Errors are assumed to have a normal distribution.
3. Errors are assumed to show homescedasticity.
   NB: errors are the difference between the observed and predicted scores.

Question 32

One of the assumptions made about the errors/residuals when doing a regression between two continuous variables is that the errors/residuals will show homoscedasticity. What does this mean?

Answer 32

If errors are homoscedastic this means that the error or residual for each observed value is random, i.e. there is no pattern or relationship to the errors observed. If this was shown on a graph of predicted values and errors then there would be a straight line going across the graph i.e. x cannot predict y.

Question 33

What is the opposite of homoscedasticity?

Answer 33

Heteroscedasicity - in regards to errors this would refer to a pattern in the errors.

Question 34

Is the standarized slope coefficient the same as Pearson’s r?

Answer 34

Yes.

Question 35

What is r-squared?

Answer 35

R-squared is Pearson’s correlation coefficient (r) squared and is reflective of the amount of variability in the outcome variable that is due to the relationship with the predictor variable. So if the correlation between weight and height is 0.8 then r-squared would be 0.8x0.8, which is 0.64. This means that 64% of the variation in weight is due to it’s relationship with height.

Question 36

Is correlation a form of bIVARIATE ANALYSIS?

Answer 36

Yes.
Correlation is a measure between two variables.

Question 37

If the relationship between two variables is non-linear, will a linear correlation analysis be helpful?

Answer 37

No.
A correlation analysis may not indicate that there is much of a relationship.

Question 38

What does the Pearson’s correlation coefficient r measure?

Answer 38

Pearson’s correlation coefficient is used to measure the linear association between two CONTINUOUS variables.

Question 39

Pearson’s correlation coefficient compares how much two continuous variables vary together to how much they vary separately.

Answer 39

Yes.

Question 40

Will there be a larger Sum of Squares when there is more variability between two variables?

Answer 40

Yes.

Question 41

What is the SUM OF PRODUCTS?

Answer 41

Sum of products is a measure of covariability.
SP = Sum of ( (X 0bserved - X mean)(Y observed - Y mean)

SS measures variability of a given variable
SS = Sum of (X obs. - X mean) squared

Question 42

For two variables X and Y, if X = Y then does SP = SS?

Answer 42

yeS.

Question 43

Does Pearson’s r = covariability of X and Y/variability of X and Y separately?

Answer 43

yes.

Question 44

How to identify whether a score should be considered an extreme score?

Answer 44

An extreme score tends to be followed by a less extreme score.

Question 45

For correlation, what is the null hypothesis?

Answer 45

That the correlation between two variables in the population will be zero.

Question 46

What is the degrees of freedom for correlation?

Answer 46

df = n-2

Question 47

If correlation refers to the association between two variables, what does regression refer to?

Answer 47

Regression refers to the description of that association and can be used as a predictive measure.
In other words it is the equation for the line of best fit of the association between two variables.

Y = a + bX + (error)
a = intercept
b = slope

Question 48

For the error in regression models, what are its qualities?

Answer 48

Normal distribution.
Homoscedastic.
Independent.

Question 49

How is the line of best fit (regression model) determined?

Answer 49

Through the LEAST SQUARES SOLUTION.

Question 50

Is the slope of a correlation the same as the correlation?

Answer 50

No.

Question 51

In regression we always need to report the variance explained (R-squared).

Answer 51

R-squared, or variance explained, is the proportion of variation in the outcome variable explained by variation in the predictor variable.
If all variation in one is explained by the other then R-squared would = 1.
If R-squared = 0.23, then 23% of the variation in the outcome variable is explained by the variation in the predictor variable.

Question 52

Does the ANOVA F test tell us whether the variance explained is significant or not and therefore whether the regression model is of use or not?

Answer 52

Yes.