Moderation, Logistic Regression, Mixed ANOVA

Question 1

What kind of relationship do we look for in a moderator

Answer 1

Here, we’re interested in whether the relationship between our predictor (X) and outcome (Y) is affected by the moderator (M).

Question 2

What are 4 different changes in relationship with a moderator

Answer 2

The relationship is sometimes smaller
The relationship is sometimes larger
The relationship sometimes disappears
The direction of the relationship changes

Question 3

What is a main difference between mediation and moderation

Answer 3

Contrary to mediation, which shows how a predictor works, moderation shows whether or when it works.
You could also say it shows us under what conditions we can expect a relationship

Question 4

What is the difference between mediation and a third variable problem

Answer 4

In a third variable problem, the primary predictor does not predict the mediator, but the opposite. The mediator is predicting both the main predictor and the outcome.

Question 5

What is some language showing you have a moderation

Answer 5

“Only when” “Sometimes” “It depends”

Question 6

How does a moderator impact the relationship of the predictor to the outcome

Answer 6

Depending on the level of the moderator, the beta will increase or decrease in strength

Question 7

Which type of variable is easier to visualize in a moderation analysis

Answer 7

categorical moderator (as opposed to continuous)

Question 8

What extra step do you need to do to calculate a moderation analysis, as opposed to a regular 3-variable mutliple regression

Answer 8

We need to add an interaction. The model we run includes 3 predictors instead of 2, but the third is just the multiplication of X by M

Question 9

Which elements are part of the moderation model

Answer 9

1. Predictor
2. Moderator
3. Interaction: Predictor x Moderator
4. Outcome

Question 10

What is an important first step to run a moderation analysis

Answer 10

All predictors need to be centered before conducting your regression.
Centering means subtracting the Mean from each observed value.
Conveniently, standardized variables are automatically centered
The interaction term must be calculated using the centered predictors.

Question 11

How can you tell if you have a significant moderator, and what should you do next

Answer 11

If the interaction term beta is significant, then we have found moderation.
You will get three betas: 2 predictors + Interaction
Follow it up with a simple slopes analysis

Question 12

What are the simple slopes plot, how should you interpret it

Answer 12

This plot allows us to visualize the effect.
It will give the average estimate, the low (-1SD) and the high (+1SD) estimate as well as p-values. If the p-values are significant, you can determine which level of X is different form the average.

Question 13

What are main features of logistic regression

Answer 13

1. Outcome we’re interested in is categorical.
2. Doesn’t require straight lines
3. Can use categorical and continuous predictors

Question 14

What is a binary logistic regression

Answer 14

The outcome has only two possibilities

Question 15

What is the goal of a logistic regression

Answer 15

Instead of focusing on the amount of variability in the outcome that is explained, we’re trying to predict which category participants fall into

With binary logistic regression we make a model to predict which of the outcome variable’s two categories each participant falls into, and then check that model against the observed outcome.

Question 16

What are the minimum and maximum value of logistic regression

Answer 16

Probability ranges between 0 and 1.

You can only have zero if the bottom of the equation reaches infinity, which won’t happen. But in theory, it could reach zero.

Question 17

What statistics tell us if our logistic regression is significant

Answer 17

Our overall model significance test isn’t an ANOVA, but rather a χ2 test (chi squared); this will tell us whether our R2, or accuracy, is high enough

We also want to know how much each predictor contributed to this model accuracy (beta, or other statistics)

Question 18

What should you do if a predictor is not significant in your logistic regression model

Answer 18

Remove it.

Just like linear regression, it’s best to use the simplest model you can get away with. That is, without removing any significant predictors.

This means we only keep predictors if they have explanatory benefit

Question 19

What happens if only one of the categories of a predictor has a significant beta?

Answer 19

if one of the levels of the categories in that predictor is significant, it justifies using the entire predictor. Not all levels have to be significant

Question 20

What can you use to make sure the predictor you removed was not significant

Answer 20

The hierarchical model teardown works well:
First fit a model with all predictors, then remove any that don’t contribute significantly

In jamovi we need to do this backwards, to get the p –value of the model change (to verify we didn’t remove too much)

Question 21

Name the 5 assumptions of logistic regression model

Answer 21

Complete Predictors
No Complete Separation
No Overdispersion
Not too much Multicollinearity: continuous predictors only
No Influential Outliers: continuous predictors only

Question 22

Explain complete predictor

Answer 22

We need data from all categories, for categorical predictors
We need the full range of responses for continuous predictors

Question 23

Explain complete separation

Answer 23

Complete separation is when the outcome is perfectly predicted. Complete separation makes it impossible to select a single wellfitting model (it’s like not being able to calculate a line of best fit) because there is a horizontal gap between the observations

We need to see some horizontal overlap between the high and low probability observations.

Question 24

How can you assess if there is complete separation

Answer 24

You can use a Descriptives analysis to determine whether this is a problem.
You need to examine the range (Minimum & Maximum) of scores for each predictor, separately for people in the DV = 0 and DV = 1 categories
Assuming you coded them as 0 and 1, of course

Go to Exploration Descriptives
Add any predictors to the Variables box
Add your outcome variable to the Split by box

Question 25

Explain overdispersion

Answer 25

The variance is larger than expected from the binomial distribution.

Essentially, there are too many observations in one condition. It’s only relevant when you have more than one predictor
This would be the same as a violation of the assumption of Independence of Errors in MR – questionable model p -values

It’s still very unlikely to be a problem as with a large sample size, it is very unlikely that our significance would change enough to change the results

underdispersion is also possible, but even more unlikely

Question 26

How can you check for multicollinearity in logistic regression

Answer 26

While we could just do a regular linear regression to check for multicollinearity problems, jamovi also has this option under the Assumption Checks
It does the exact same thing, and gives you VIF/Tolerance
Assuming you are using continuous predictors

Question 27

How can you check for outliers in logistic regression

Answer 27

For influential outliers we need to do a linear regression, so that we can do Mahalanobis and Cook’s distance checks
We can’t do it 100% correctly (except for Mahalanobis – why?) but it will be approximately right

Question 28

Which statistics could you report from a logistic regression

Answer 28

Deviance (aka -2LL) : higher values mean better model fit
Model prediction %
Cox and Snell R2
Nagelkerke R2
McFadden’s R2
Odds Ratio - SPSS called this the Exp(B), just FYI

Question 29

Explain the Odds Ratio

Answer 29

The Odds Ratio is the change in the odds of the higher-numbered outcome occurring given a 1 unit change in the predictor.

It is, mercifully, similar to a Beta in that it can show positive and negative relations – but…
If OR is > 1 then the relation is positive
If OR is < 1 then the relation is negative
The OR can’t go below 0, but can go as high as infinity
there is the same amount of space between 0 and 1, then between 1 and infinity using Odds Ratio

1 is our baseline point, as opposed to 0

Question 30

What is the Odds Ratio formula

Answer 30

odds after 1 unit change/original odds

Original odds: P(event)/P(no event)

Question 31

How can you interpret a positive Odds Ratio

Answer 31

If OR = 3.42, then the probability is 2.42 or 242%

Question 32

How can you interpret a negative Odds Ratio

Answer 32

If OR = .292, then you must convert it to positive -1/.292 = -3.42. Then the probability is -2.42 or -242%

Question 33

What is the standardized coefficient of a logistic regression and why

Answer 33

there are none

Unlike mediation, that’s not something we’ll try to work around
It’s probably best for the situations where you’re likely to use logistic regression
because we are accepting categorical variables, standardizing variables doesn’t necessarily make sense

Question 34

Describe the mixed ANOVA

Answer 34

A Mixed ANOVA is one that includes at least two different kinds of independent variable and only one dependent variable. There’s nothing theoretically new about this ANOVA.

You will have at least 2 main effects, and at least 1 interaction

All IVs are categorical
At least one uses the same participants (repeated measures)

At least one uses different participants (independent measures)
Any DVs are continuous

Question 35

What kind of effect are you expected to find in a mixed ANOVA

Answer 35

In a Mixed ANOVA design, you get both main effects and interactions among your between- and within-subjects IVs.

Main Effects: An F for each IV
E.g., First Experience of Task (between) & Type of Experience (within)
Interactions: An F for each possible IV combination
E.g., First Experience x Type of Experience (2-way interaction)

Question 36

What should you care the most about when interpreting the result of a mixed ANOVA

Answer 36

Interactions tell you there is a complex story to tell, where your outcome (degree of DV) depends on knowing where a person falls on all IVs involved. If present, they’re all you need to care about.

Question 37

Once you find significant F statistics in a mixed ANOVA, what should you do next

Answer 37

To properly interpret a mixed ANOVA, you would follow up on your main effects and interactions just as you would for any other ANOVA – with simple effects analyses.

1. Post-hoc tests are often the best method (recall: Tukey)
2. Paired-samples t-tests could be required for repeated-measures IVs. To check an interaction, you may need to filter your file to perform these tests on only specific groups
3. Independent-samples t-tests could be used for between-subjects IVs
4. Marginal means tables are important for making figures

Question 38

Why do we need to center our variables in the moderation analysis

Answer 38

In order to calculate the interaction variable, our data must be centered.
If it is not, we will have too much collinearity violating our assumptions for this model.

Why does it fix the problem? We don’t know, but the statistics demonstrate it

Question 39

When should you worry about overdispersion

Answer 39

It’s only relevant when you have more than one predictor

Question 40

How can you test for overdispersion

Answer 40

You can’t, but it is highly unlikely to be problematic. Especially with a large enough sample size

Question 41

What type of variables for IV and DV do you need in mixed anova

Answer 41

All IVs are categorical

Any DVs are continuous