Mixed ANOVA

Question 1

Mixed Designs

Answer 1

• Mixed designs occur when we use a mix of between-subjects and within-subjects designs
  
  • One or more within-subjects IV(s) and one or more between-subjects IV(s)
• There are many occasions when variables need to be between-subjects:
  
  • Quasi-experimental designs, such as when we have existing groups
  • For example, a clinical group (e.g., with depression) and a control group
• There are many occasions when variables are best suited to be between-subjects:
  
  • For example, drug trials where the treatment can be long term, so only one group receives the treatment and the other serves as a control
• There are many occasions when variables are best suited to be within-subjects:
  For example, assessing the effectiveness of a drug through a pre-test assessment and a post-test assessment

Question 2

Quasi Experimental Designs

Answer 2

• Quasi-Experimental Designs can make it difficult to establish causation. Any effects in the DV that we attribute to the IV could actually be caused by:
  
  • The IV that we think/hope is causing the effect
  • A confounding variable that people in different groups systematically differ in
  • Often difficult in clinical cases where condition co-morbidity is common; is the difference based on our condition of interest, or a condition that commonly co-occurs with it?

Question 3

how to control quasi designs?

Answer 3

• One way to try and control for this is through a matched design
  
  • In a matched design, you match participants in your control group on critical factors that you think may influence your DV
    For example, in developmental studies, children are often matched on age

Question 4

pre/post test designs

Answer 4

• Perhaps the most common mixed design in psychology research
• We have some treatment that we want to assess the effectiveness of (between-subjects IV), and some (continuous) way of measuring the effectiveness (DV)
• However, there’s always a chance that despite our random assignment, the groups differed before our treatment
  
  • Perhaps unlikely, but definitely possible, especially with small samples
• We can remove this possibility by testing people both before they receive the treatment, and after they receive the treatment (within-subjects IV)
• This mixed design removes a key potential confound of the purely between-subjects design, by simply adding another measurement point
  
  • Our interest is then primarily in the interaction between the Ivs
• For example, we want to see if a group who receives CBT has lower depression scores (Beck Depression Inventory) compared to controls who receive no treatment

Question 5

interactions

Answer 5

• Interactions become particularly important in mixed designs
• When dealing with pre/post-test situations, our key interest is in the interaction term
• Different interactions can also mean different things
• It’s really important to know how to interpret these correctly if you want to make robust inferences for mixed designs

Question 6

Mixed ANOVA assumptions

Answer 6

• The standard ones:
  
  • DV should be at the scale level
  • Data should be normally distributed
• Then, kind of a “worst of both worlds” situation, and then some…
• The curse of between-subjects ANOVA:
  
  • Equal Variances
• The curse of within-subjects ANOVA:
  
  • Sphericity
• The curse of daring to mixed between-subjects and within-subjects:
  
  • Equality of covariance matrices

Question 7

Equality of covariance matrices

Answer 7

• A really fancy way of saying that the correlation between measurements of the within-subjects IV should be the same across the groups of the between-subjects IV
• … I guess that itself is a really fancy way of saying that the equal variances and sphericity assumptions have merged into a horrible beast?
• Also not a particularly good way to test this (something called Box’s test, but it’s overly sensitive), or particularly great alternate strategies if it’s violated…
  
  • However, linear mixed models do not make assumptions about the covariance matrix (hence no sphericity), so we can dodge this bullet too

Question 8

what to do in R

Answer 8

• Step 1: Install (if necessary) and load in necessary packages
• Step 2: Load the data
• Step 3: Look at and format the data
• Step 4: Test assumptions
• Step 5: Run the “ANOVA” (linear mixed model)
• Step 6: Get the important information
• Step 7: Run post-hoc tests
  Step 8: Get the important information

Question 9

summary

Answer 9

• Mixed designs are an example of a complex design which incorporates both between and within subject variables
• Mixed designs combine the advantages of within and between subjects designs!
• … they combine the disadvantages of within and between subjects designs :’(
• We should always consider whether variables are best suited to being within subjects or between subjects (when we have a choice, e.g., not in quasi-designs)
  Like all Complex ANOVA, we need to break down significant main effects with 3+ levels, and significant interactions, to understand where the differences are