General linear model

Question 1

General linear model

Answer 1

all about expressing the relationship between variables

For example…
What is the relation between a test score and the grouping variable
What is the relation between a pre-and-post-test measure

Question 2

GLM Stat tests

Answer 2

T-tests
ANOVA, ANCOVA, MANOVA, MANCOVA
Correlations (pearson and spearman)
Linear regressions and multiple regressions
Goodness of fit test/chi squares
Machine learning and prediction models

Question 3

GLM equation

Answer 3

using this equation, we can predict the outcome variable Ŷi for participant i, as long as we have the X value for participant i

Ŷi = b0 + b1xi + ei

Question 4

Ŷ

Answer 4

Ŷ is the estimate of the observation outcome (Y) ie. represent the estimated DV

Question 5

b0

Answer 5

b0 is the intercept of the regression line (where is crosses the axis)

Question 6

B1

Answer 6

B1 is the the slope of the regression line

Question 7

xi

Answer 7

xi is the observation of the predictor (X) ie represents the IV

Question 8

ei

Answer 8

ei Is the residual error term, which is the difference between observed and predicted Y

Question 9

i

Answer 9

i stands for the participant whose data is being used

Question 10

Correlation

Answer 10

a standardized measure of the linear relation between two variables

X and Y are interchangeable

Question 11

r-value

Answer 11

The correlation is represented by an r-value that can take any value between -1.00 to +1.00

The numerical value represents the shape of the correlation, the positive or negative represents the direction

A 1 is a perfect line, while a smaller value like 0.2 will be unfocused along a line

positive means when the IV increases so does the DV (and the oppositve for negative)

Question 12

Common correlation interpretations

Answer 12

For absolute correlation values (positive or negative), common interpretation:

= 0.00 no relation, entirely random
0.01 to 0.30 weak
0.30 to 0.50 moderate
0.50 to 0.99 strong
= 1.00 perfect, identical

But these rules are arbitrary and should be based on the context of the study

Question 13

Anscombe quartet

Answer 13

Idea that graphed data can look totally different but have the same summary statistics

Question 14

Ordinary least squares regression

Answer 14

The general linear model tries to create or fit a line (line of best fit) through the datapoints that is as close as possible to every datapoint

This is done by minimizing the squared distance between the line and each point, which is why it is called “Ordinary least squares regression”

By default its estimates come out unstandardized ie. using the units of the original variables

Question 15

Coefficients for ordinary least squares regression

Answer 15

For ordinary least squares regression: The estimated regression coefficients (b0 and b1) are the those that minimise the sum of the squared residuals.

Take the distance between a datapoint and the fitted line
Square that distance

Repeat for all datapoints, and sum up all these surfaces

Find the line where this combined surface is the smallest.

Question 16

Unstandardized

Answer 16

Unstandardized coefficients are based on raw data and one unit changes in the IV

Unstandardized estimates more intuitive, but can’t easily be compared across different kinds of measurements

“For every 1 min difference in average exercise per day, there’s a 0.017 difference in BMI” (unstandardized)

Question 17

Standardized

Answer 17

Standardized data is on analyzed data/standard deviations

Standardized estimates are less concrete, but can be compared across different measurements; can use the correlation “rules of thumb” we discussed above

“For every 1SD difference in average exercise per day, there’s a 0.176 SD difference in BMI” (standardized)
OR
“Average minutes exercise per day 3% of the variance in BMI” (standardized R2)

Question 18

Multiple/linear regression model formula

Answer 18

When including several predictors: Need Multiple regression model

Ŷi = b0 + b1xi + b2xi +ei ,

Can go on adding as many predictors as makes sense
Ŷi = b0 + b1xi + b2xi + b3xi … +ei ,

But, instead of a singular line, we are now trying to create a plane within a 3D space that still minimizes the distance between observed data points

Question 19

Controlling/Adjusting/Partialing out in Linear Regressions

Answer 19

All refer to the same process of when having multiple variables in your regression

If you control outcome Y for predictor variable X, then check the association between variable Z and outcome Y, you’re asking: “what would be the Y~Z relation in a sample where everyone had the average level of X?”

This is not MAGIC

Predictions from regression models, even if “controlled” don’t suddenly make associations causal

All depends on where your data came from

If they’re from a randomised experiment, causal conclusions might be justified
If they’re from an observational study, probably no

Question 20

(Multiple) regression assumptions

Answer 20

Normality (of residuals) (-> if you were to plot the residuals you would see a normal distribution)

Linearity (-> associations between X and Y are linear, aka constant)

Homogeneity of variance (of residuals)

Uncorrelated predictors (-> no collinearity)

Uncorrelated residuals (-> no effect of another unmeasured variable)

No highly-influential outliers

Question 21

T-tests and GLM

Answer 21

Subtype of GLM, equivalent of simple linear regression

Think of the intercept as the mean of group 1

And the slope as the distance from the intercept to the mean of group 2 (the black line)

Ŷi = b0(mean_group1) + b1(mean_group2–mean_group1)xi + ei

Question 22

ANOVA

Answer 22

Comparing more than 2 group
ANOVA (Analysis Of VAriance) is a kind of general linear model that only has categorical predictors

Even though it’s called analysis of variance, it’s actually mainly interested in differences between means

A one-way ANOVA, comparing ≥ 3 means, is equivalent to a multiple regression model

The ANOVA’s test statistic is the F-ratio – the ratio of variance explained between the groups to that explained within them

Question 23

R: Get r-value/cor coeff

Answer 23

cor.test(dataset$variable_1 , dataset$variable_2)

Question 24

R: Plot correlation

Answer 24

plot(dataset$variable_1 , dataset$variable_2)

Question 25

R: Plot with line of best fit (unstandardized)

Answer 25

ggplot(dataset, aes( x = IV, y + DV)0 + geom_point() + stat_smooth(method = lm)

Question 26

R: linear model

Answer 26

new_name <- lm(IV ~ DV , data = dataset)
summary(new_name)

Question 27

R: Find what type of data

Answer 27

class(dataset$variable)

Question 28

R: Convert to factor

Answer 28

dataset$variable <- factor(dataset$variable)

Question 29

R: Multiple regression

Answer 29

lm(variable_1 ~ variable_2 + predictor, data = dataset)

Question 30

R: Remove outliers

Answer 30

dataset$variable[dataset$variable>or<#] <- NA

Question 31

R: Multiple regression with interaction

Answer 31

name <- lm(variable_1 ~ variable_2 (+ predictor if necessary) + interactionv1:interactionv2, , data = dataset)
summary(name)

Question 32

R: Plot interaction

Answer 32

interact_plot(name, pred = IV, modx = DV)

Question 33

R: ANOVA

Answer 33

aov ()