Discussion Meeting: Semantic Memory

Question 1

Name the strengths of the article by Smith et al. (2019)

Answer 1

1. Large sample size
2. Homogeneous timeframe (limits extraneous effects of time of day)
3. They provide a biological basis for their research
4. Take psychological AND physiological measures of stress
5. Chose 122 most easily remembered questions (limits variability in influence of prior knowledge)

Question 2

Name the limitations of the article by Smith et al. (2019)

Answer 2

1. Female overrepresentation
2. No brain activity recordings
3. No clear distinction between stress and anxiety (–> low corr. between physio and psycho stress tests)
4. No concrete exclusion criteria

Question 3

What are the limitations of the article by Heyman et al. (2022)?

Answer 3

1. Multiverse analysis is an not all-powerful method
2. DoF can still be manipulated depending on what paths you want to analyse
3. Without preregistration you can still choose which results you want to demonstrate in your paper
4. Original paper didnt consider many confounds → thus Heyman et al. couldnt do so either
5. If original research was faulty (e.g. insufficient power, faulty selection, non-examined confounders etc), then Heyman et al. using the same faulty data cannot conduct more reliable analyses
6. Only presented p-values, not other summary data

Question 4

Smith et al. (2019) used dummy coding, while Heyman et al. (2022) used effect coding for their variables. Why is Heyman using a different coding scheme NOT a limitation of their study?

Answer 4

because they DID think about it but dummy coding is the improper way to begin with so it doesnt make sense to analyse data in a knowingly wrong way

Question 5

What is a limitation of Multiuniverse Analysis in respect to interpretation of findings?

Answer 5

the variability in p-values raises questions as to what is correct, what to believe, and how can we know which methods are better

Question 6

What is a limitation of MultiUniverse Analysis with respect to the data being analysed?

Answer 6

if there is a problem with the data you’re analysing, maybe there’s a confound you didnt notice and you cannot correct for the issue because the same dataset is used in all MUA analyses

Question 7

based on both studies, what elementswould you include in your new and improved experiment to study effect of stress on semantic memory?

Answer 7

1. Time pressure induces stress even if in control condition → remove time limit
2. Forced responses → cant leave blank responses
3. Operationalise psych stress as the change in psychological stress level
4. Separate analyses with physiological + psychological stress
5. Equivalent gender groups
6. Question content for general knowledge test (relates to “men scoring higher on average on test compared to women”)

Question 8

What are the limitations of Smith et al. (2019) according to Heyman et al. (2022)?

Answer 8

1. Potential false positives: possible that Smith finding was a false positive considering that most studies find stress to NEGATIVELY affect memory, not POSITIVELY affect it
2. Treatment of omissions → various other ways they couldve been considered
3. Psych stress measures only used as supplementary to physiological → not both operationalised as difference between T1 and T2
4. Psych stress measures depend on introspection which some may lack
5. Coding scheme: used dummy coding instead of affective coding → linear regression is easiest when looking at effect of continuous on continuous variable. When including categorical predictors u can use dummy coding. The coding has an effect on how you interpret beta → interpretation assigned to beta does not correspond to the one in the model they used. Effect coding is better

Question 9

What could be a reason that the psychological and physiological stress measures have a low correlation with each other? Correlate this to the findings about semantic memory

Answer 9

If the psych measure is a measure of ANXIETY instead of stress, it makes sense that it doesnt influence SM. However, if it does indeed measure stress, it s problematic that psych stress measures didnt correlate sign with SM changes, but physiological measures did

Question 10

What did Smith et al. attempt to demonstrate with their study?

Answer 10

that acute stress enhances general-knowledge semantic memory

Question 11

In an older study, Merz et al. (2016) reported that higher cortisol reactivity interferes with SM retrieval in a sentence verification task, while Smith et al. (2019) found the opposite. What 3 reasons do Heyman et al. (2022) for these different findings?

Answer 11

(a) Merz could’ve been more complex task
(b) Merz = RTs & Smith = accuracy (possible speed-accuracy tradeoff)
(c) Smith could’ve found a false positive